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CHAPTER I: INTRODUCTION 



It is eonanon knowledge that growing student enrollments and the 
dearth of professorial talent have forced many colleges and universities 
to seek new ways of meeting instructional obligations. Some have chosen 
to establish new liaison groups ) such as the Educational Development 
Program at Michigan State University, charged with the task of facili- 
tating more effective ways of meeting growing instructional demands. 
These demands frequently result in the use of the newer media of commun- 
ications. With few models for guidance, the newly- formed groups have 
been searching for a viable organizational structure and accompanying 
set of operating procedures * 

It was the awareness of this need that prompted the 1963-65 
study at Michigan State University, A Procedural and Cost Analysis 
Study of Media in Instructional Systems Development . A majnr portion 
of- this investigation was given over to the development and field 
testing of an organization and operation which could serve as a model 
Instructional development system. The development 'model consists of an 
organizational scheme which differentiates substantive personnel 
specialties and the steps of instructional development c The decision 
sequence and respective information needs were derived in the analysis 
of actual course development work by faculty members and specialists 
over a five-year period. 

Recognizing the pcssibillties of further model refinement and 
dissemination, the findings were incorporated into a project at 
four universities, the outcomes of which are described in this report. 

In what might seem a unique move. Investigators at Michigan State 
University heeded advice from Brickeil, Rogers, Miles, et al., which 
boils down to the statement of Brickeil: 

The most effective way to convince a school staff that 
it should adopt a new program is to let it observe the 
successful new program in action. Nothing persuades like 
a visit. 

This major demonstration program enlisted the aid of Michigan 
State University, University of Colorado, San Francisco State College, 
and Syracuse University. These institutions exhibited a readiness and 
willingness to move into full-scale instructional planning and had 
expressed a need for an organizational structure and accompanying 
operating procedures -- a development system. By incorporating the 
instructional development procedures evolved in the earlier United 
States Office of Education Study, these schools not only served as 
working models for Instructional systems development for other colleges 
and universities in their region, but also fed back data as to the 
feasibility and reliability of the trial system. 
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Specifically, the project *s objective^ were: 

1* To select and orient innovative institutions which are to 
serve as working models of systematic instructional planning 
which utilises the entire range of newer media* 

2. To evaluate this demonstration inethod of disseminating infor- 
mation by gathering data relevant to the adoption of the 
developmental system from visiting institutions. 

3. To evaluate the (a) feasibility, and (h) reliability of this 
developmental system vithin institutions, first (hopefully) 
with a coitaion course, ?nd second, over other key courses. 

4. To refine the system's organisation and operating procedures, 
both in light of the above evaluation and the re-evaluation of 
a second trial. 

5. To devise and build in a nieans for generating research hypo- 
theses within the developmental system. 

6. To prepare a prototype list of research h 3 rpotheses that have 
been generated by the working demonstration models. 

7. To sponsor major ^n^osia at DAVX and AASA conventions, the 
participants being members of the demonstration teams and 
project staff. 

8. To aid the participating demonstration institutions in preparing 
suxnmary presentations which capitalize on the advantages of 
media application. 

9. To produce a summary document which (a) describes the major 
steps taken by the demonstrating institutions in implementing 
a systematic approach to instructional planning; (b) contains 
the final form of the developmental system; (c) includes 
diffusion data representing the number of visiting institutions 
and those which have shown evidence of system adoption; (d) 
offers a prescription for the curricula of substantive degree 
programs based upon role- task specification of system specialists; 
and (e) presents comparative cost data for instructional 
systems development at each of the demonstration institutions. 

An evaluation dimension was Introduced to the model trials by the 
assignment of several experts from outside the project to appraise 
changes suggested by trial outcomes and the value of the demonstration 
method employed for dissemination. 
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The overall administration of the project was the „„ 

of the coordination Unit located on the Michigan State campus. Addition- 
al advlsemLt and guidance was obtained from a Project Adviso^ C^ittee 
composed of university administrators with considerable experience 1 

-tMnmptth-tnn . 4 ns ti^c tlonal developmeiit , and evaluation* 

The finJings presented in the report are arranged so as to 

tives of work at the U*dividual schools and the overall investigation. 

Each of the demonstration institutions describes (1) the background 
of two cases of course development, (2) the plan of action applied to 
this development, (3) the competencies of the specialists involved, (4) 
the actual sequence of events, and (5) the outcomes of the development 
activities* They also offer some conclusions derived from their 
experiences . 



The members of the Evaluation Team offer opinions and observations 
on the same aspects of the development procedures, as derived from 
disctissions with development teams, examination of audio tape recordings, 
and other sources of information. Their work was aimed at increasing 
the objectivity of the evaluation of the development model and dissemina- 
tion activities across all four institutions. 



The Coordination Unit report consists of generalizations and 
recommendations for study vjhich should be implemented in the future. 

In addition, some general information is offered on the costs associated 
with the type of instructional development demonstrated in the Project. 
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CHAPTER II: METHOD 



Testing A Model 

The design of this study was guided by the premise that the 
procedures of media innovation in college courses and instructional 
development could be Improved and disseminated through a cooperative 
testing program at four universities, which had history of success in 
such work. The object of the tests was a model of steps for media 
innovation in college Instruction, evolved in a prior media study at 
Michigan State University. Figure 1 is a flow chart depicting the 
course development procedures and their sequence. The tesults of these 
tests were used to refine and reshape the mbdel, as Well as offer rare 
opportunities for assorted collega authdrities to gain a greater under- 
standing of the complexities of the development problem and the compe- 
tendies required* 

Selection of Participants 

The key to conducting the investigation on a large scale and 
restructed time schedule was the selection of four institutions, 
which met several criteria: 

1. Commitment to immediately initiate a coordinated Instructional 
development design in two academic areas, each enrolling a 
minimum of 500 students per year. 

2. A willingness to employ the model development system in 
their design and to maintain appropriate information feed- 
back records. 

3. Availability of necessary specialists and personnel* 

4. Agreement to conduct clinics for visiting educators. 

5. Central location in a national geographic region* 

The 4ower limit for student enrollment in the courses to be 
treated was set to insure the selection of an Instructional program 
which had a greater likelihood for media usage. Also, it was 
anticipated that large enrollment courses Involved more university 
staff in the decisions guiding its conduct. 

The investigators hoped that of the two courses, one could be 
in a common subject area across all four institutions. The shortness, 
of lead time and conflicting schedules prevented this arrangement. 

The demonstration institutions alleviated the time limitations by 
selecting courses in departments which exhibited . a readiness and 
willingness to cooperate in the development task outlined in the 
proposal . 
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Figure 1 



A FLOW CHART* OF PROCEDURES FOR ANALYSIS OF INSTRUCTION AND 
IMPLEMENTATION OF NEWER MEDIA OF COMMUNICATIONS 



KEY 

I — Instructor 
IS — Instructional Specialist 
ES — Evaluation Specialist 
MS — Media Specialist 



Various curricuium committees 
state goals in broad terms 



r 



( DETERMINE BROAD EDUCATIONAL GOALS I 

COLLEGE-SCHOOL-DEPT.-COURSE 1 
y 

fsEGINl 

I GATHER INPUT DATA~1 

I 

A SPECIFY ENTRY AND TERMINAL BEHAVIOrI ES joins I and IS to assist in 

description of specific objec- 



I meets with IS 

I assesses course limits, 
number of students, available 
finances, materials, etc. 



DEVELOP RATIONALE FOR 
PRE AND POST EXAMS 



I and ES develop testing situations 
which measure defined behavior 



I 



-A TOTAL INPUT DATA COMBINE^ 



I 



1 PLAN STRATEGIES 1 


DEVELOP TEACHING EXAMPLES 






OF DETERMINED CONTENT 



size, teacher student ratio 
contact, communication methods, 
experience factors, etc., based on 
theory of instruction 

rCHOOSE REPRESENTATIVE INFORMATION F 



tives, content, and behavior 



I and IS con^ile completed 
input information 



I, IS, MS, and other resource persons 
decide on information sources and 
exemplars 



1 



ORMS I 



I and MS determine best models based 
on perception and learning theory 



I DECIDE ON TRANSMISSION VEHICLES | 



I and MS determine which 
of various media is 
called for at points 
within system 



COLLECT, DESIGN. PRODUCE SPECIFIED MEDIA I 
I DRY RUn|tHROUGH~I 



DEVELOP EVALUATION 
INSTRUMENTS WITH STUDENT 
DATA AS WELL AS MEDIA 
NFORMATION . 



FIELD TEST SAMPLES 
WITH STUDENT CROUP 



I 



I, MS, and IS conduct representative 
dry runs of system packages 

I, IS, and MS check feasibility of 
system with live audience and 
related test samples 



I LOCATE AND (^ORRECT FLAWS 1 
— M APPLICATION TO COURSE | 



X 



EVALUATION AND RE-CYCLE 
TO REFINE AS NECESSARY 



*Note: Information feedback loops 
have been deleted from this 
illustration. 
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Each institution designated a campus development team leader 
responsible for the local conduct of the study, including the dissem- 
ination aspects. This staff member also served as the instructional 
specialists in the development procedure#. His initial responsibility 
was to select the best qualified persons he could obtain at his 
institution for the roles of evaluation specialist and media special- 
ist in rounding out the development team. 

The members of the overall project evaluation team were selected 
by the Coordination Unit on the basis of experience and qualifications 
in media innovation, instructional development, and dissemination 
studies. 

The other participants and their relationships are depicted in 
Figure 2, the Project Organization Chart. 

Orientation of Participants 

The physical distance separating project operations made it 
essential to provide periodic face-to-face discussion among partici- 
pants. These information exchanges were accomplished at spaced 
orientation and project review meetings. Wlxenever possible, they 
were held in conjunction with professional conferences at which 
dissemination opportunities also existed. The initial orientation 
meeting was conducted in September, 1965, at Michigan State and was 
designed to acquaint development team members with the details of 
Instructional development activities to be undertaken at their 
institutions. The orientation Included: 

1. A presentation on the structure of the model development 
system. 

2. A simulated run-through of the procedures j demonstrating: 

a. Specific decision points and the rationales leading to each. 

b. Instruction in the use of the information transmittal forms. 

c. The sequential mature of decision-making. 

d. Specialist's domains and functions. 

e. Use of support systems. 

3. Instruction in the completion of evaluation forms to be sent 
to the evaluation team members . 

4. Details on the administration and function of the proposed 
clinics and subsequent follow-up data collection. 

5. The demonstration teams' role in dissemination symposia. 



Figure 2 

PROJECT ORGANIZATION CHART 



INSTRUCTIONAL SYSTEMS DEVELOPMENT; A DISSEMINATION AND EVALUATION PROJECT 

1965-1962 

Instructional Media Center 
Michigan State University 



Evaluation Team 
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Analyst 

Diffusion Activities 
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Media Development 
Analyst 



Project Advisory 
Committee 
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Project Director 
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University 

Development Team 
Leader 

Media Specialist 
Eval. Specialist 
Instruc. Specialist 



Syracuse University 



Development Team 
Leader 

Media Specialist 
Eval. Specialist 
Instruc. Specialist 
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San Francisco 
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Development Team 




Development Team 


Leader 




Leader 


Media Specialist 




Media Specialist 


Eval. Specialist 




Eval. Specialist 


Instruc. Specialist 




Instruc. Specialist 
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Progressive review and revision of Project activities were accomplished 
through meetings of development team leaders, the Evaluation Team, and 
members of the Advisoty Committee at approximately six-month intervals. 

Analysis of Development Procedures Trials Data 

The investigators regarded the transactions among specialists and 
faculty members participating in instructional development as the 
primary source of data for analysis. Collecting this information was 
obviously a formidable task. The most practical means for capturing 
the details of development appeared to be recording these deliberations 
on audiotape. One institution employed videotape during a limited 
period for later use in dissemination clinics. The recording of dis- 
cussions met with few objections from faculty members and tended to 
remove anxieties about problems their use with faculty members might 
produce. 

Other sources of data were: (1) summary logs kept by development 

team specialists on their activities; (2) progress estimate and 
reaction forms filled out by participants before and after each deve- 
lopment session; and (3) reports submitted by development team 
leaders quarterly and at the review meetings. 

The recordings of the development sessions, as any analyst of 
group interaction can attest, provided a virtual flood of uncoded 
information. Anticipating this problem, the Coordination Unit 
undertook a preliminary abstracting of the tapes before sending them 
to the Evaluation Team. The Project Advisory Committee suggested 
that perhaps the most significant cue for linking the progress of the 
development discussions to the model procedures were the questions 
asked. Accordingly, auditioners in the Coordination Unit reviewed 
each tape, making verbatin notations of questions in the sequence 
they were asked. Other commentary was summarized and interspresed 
according to time sequence. The transcriptions, tapes, and log 
reports were sent to the Evaluation Team members for review and 
analysis . 

Members of the Evaluation Team were asked to study the written 
and taped records of development procedures and generate responses 
to these project concerns: 

I. Measuring Procedures Effectiveness 

A. What is the feasibility of the hypothetical model for 
media innovation and instructional development when 
measured by the following criteria? 

1. General fit in university setting : 

You observe the model operating sufficiently 
trouble-free for the demonstration teams to 
^udge it acceptable for general use in higher 
education. 
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Conditional fit in university setting: 



% 

Yoii observe model usefullness varies consid** 
erably with the resources, skills, and "politics” 
of the deipcnstration institutions * Varying 
ids tttxb tors ahd instructional situations tequlre 
dhiftibg steps in tha development tfodel sequence 
dhd changing the “iniic” of specialists. 

3. unacceptable fit in university setting ; 

You observe the model was more of an obstacle 
than guide in the development activities of 
the demonstration teams. Kuch more than simple 
changes in the sequence of the model was re- 
quired. New steps were devised and introduced. 

B. What is the reliability of the model for media innova- 
tion and instructional development when measured as the 
degree of "consistency” in the kinds of activity gene- 
rated by the operation of given steps in the model? 

1. "High consistency" of outcomes ; 

You observe that given steps of the model generate 
the similar kinds of questions and development 
activities across institutions. 

2. "Low consistency" of outcomes ; 

You observe given steps of the model generate 
almost random questions and development activi- 
ties across institutions. 

IX. Procedure Refinement 

A. Interpret Demonstration Team Experiences 

1. Development step sequence ; 

Do the experiences of the demonstration teams 
suggest alterations in the hypothetical model 
development steps sequence? 

If yes: a. What leads to this conclusion? 

b. Do the changes constitute a major 
or minor overhaul? 

c. Is the verdict for change cornoon in 

all of the test institution experiences? 
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d. Are change recorm&andatlons for 
different Institutions in contra- 
diction to each other? 

If no: a. What further testing or treatment 

of the development sequence appears 
necessary? 

2. Develop step addition : 

What new development steps are revealed by the work 
of the demonstration teams? 



If some: 



If none: 



a. Are the new steps related to and do 
they fit the present model? 

b. Are the newly revealed steps common 
to all four test institutions? 

c. Are any of the new steps observed in 
the work of different institutions in 
contradiction? 

d. What further testing of the new steps 
do you recomnend? 

a. What rationale can you give for the 
lack of new step development? 



3. Development step refinement ; 



What refinements in existing development steps were 
revealed by the work of the demonstration teams? 



If some : 



If none: 



a. How doss the refinement differ among 
demonstration institutions? 

b. What appear to be the chief causes 
for development step refinement? 

c. Do refinements involve the deletion 
of any existing steps? 

d. What are the practical limits of step 
refinement? At individual institutions? 
In a generalized model? 

a. What reasons can you offer for the 
absence of step refinement? 



B. Mechanism for Generating Research Hypotheses 



1. What nature of methanism appears feasible from a 
review of the demonstration team experiences and how 
are the hypotheses generated? 

2. How are the steps of the development process linked 
to the hypotheses generating mechanism? 
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3. Can this mechanism or structure be modeled, either 
separately or within the development system model? 

III. Prototype List of Research Hypotheses Resulting From Trials 

A. Are these statements most useful couched in terms of 
experimental propositions, or expressed in common problem 
descriptions? 

B. Must the hypotheses deal with the specific problem in the 
demonstration institutions or can ‘hey be generalized to 
all instructional development? 

IV. Implications for System Specialists* Preparation Cu:ricula 

A. What observations in the demonstration team experiences 
support the recommendations for specialist training? 

1. Evaluation Specialist? 

2. Instructional Specialist? 

3. Media Specialist? 

B. Are these recommendations reconcilable with present-day 
training opportunities or need new discipline be created? 

C. What role does "practical experience" play in the develop- 
ment of competencies of the specialists? 

D. To what extent do unexplored research areas limit the 
competencies of the specialists? 

Dissemination Study 

Concern for judging the effectiveness of the dissemination 
activities paralleled concern for information on the worth of the 
model development procedures. The Investigators were curious about 

(1) who was influenced by the work at the demonstration institutions; 

(2) how they were affected, and (3) what they did about the information 
received. Three sources were tapped in order to obtain these 
measures; the dissemination experiences of the development teams, 
reactions of staff members on the demonstration team campuses, and 
parties outside the demonstration institution who either witnessed the 
development work or presentations of it on visits or in the clinic 
sessions. Appendix A contains an information collection foami employed 
in obtaining some of this information. The Evaluation Team examined 
these data in offering observations on dissemination mechanics and 
effectiveness. 
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A. Demonstration Mechanics 

1. Can the development model be successfully communicated 
to occasional "visitors” or is long-term observation 
required? If yes, by what means? 

2. What supporting documents or displays are necessary for 
dissemination? Do you think the process can adequately 
be "packaged" for presentation? 

3. What roles should the various specialists and the instructor 
play in demonstration? 

4. Should the "display technique" employed for communication 
with on-campus colleagues differ from that for off-campus 
visitors? 

B. Dissemination Effectiveness 

1. Who appeared to be most influenced by the procedures 
demonstrations ? 

2. To what extent were they influenced? 

3. Where do you rate the usefulness of the demonstration 
method for disseminating a procedural approach? What 
alternatives appear to be more useful? 

Costs Accounting 

It is nearly impossible to identify the total expenditures of time 
and money involved in the trials of the development procedures. Funds 
provided by the study project covered only part of the costs. The 
universities donated a good deal of faculty and specialist time and 
physical facilities in exchange for the benefits their overall organiza- 
tion would derive from project outcomes. However, in the interest of 
providing some idea as costs associated with instructional development, 
the investigators have analyzed project finances and classified time 
and money expenditures, including university-contributed resources, 
where this information was available. 
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CHAPTER III: RESULTS 



The primary purpose of this study was to influence educators 
at institutions of higher education toward employing certain proposed 
instructional development steps in the design of instruction and the 
Implementation of newer media. 

This outcome was sought through the demonstration of model 
development procedures by specialist teams in actual courses of 
instruction at four cooperating universities. 

The demonstration activities also J>rovided an excellent op- 
portunity to test the usefullness of the development steps across 
four institutions and attempt further refinements. 

The results of this investigation are reported in three parts. 
First, both the outcomes of the demonstration activities and de- 
velopment procedures refinement are reported by the teams of 
specialists from each institution. Second, four evaluators, who 
were given the opportunity to intimately observe these activities, 
report their assessment of key aspects in the investigation. 

Finally, the Project Coordinating Unit reports the approximate 
costs of the four cases of instructional development. The 
specialist teams summarize their work in the following respects: 

Report of Demonstration-Development Activities 

1. Background of Instructional development problem 

2. Steps of instructional development employed 

3. Competencies of the development specialists Involved 

4. Outcomes of the development activities 

5. Unresolved issues of instructional development 

Conclusions Derived from Demonstration-Development Activities 

1. Comparison of kind and sequence of t *elopment steps 
employed with those stated in hypothtical model. 

Estimate of the generalizability of developmental 
model to other higher Institutions pursuing 
Instructional development. 
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Project evaluators observed specialist teams' progress by 
ifgviewing audiotapes and logs of the development activities and 
periodic discussions with team members. They report their ob- 
servations in the following four papers; 

1. Analysis of the Data Secured on Actual Development 
Procedures - Dr. Phil C. Lange. 

2. Heuristics of Media Implementation in Instructional 
Systems Development - Dr. John B. Haney. 

3. Usefulness of the Development Model for Generating 
Research Hypotheses and Specifications for Specialist 
Training - Dr. John M. Gordon, 

4. Dissemination Value of the Demonstration Approach - 
br. John Childs. 

Recognizing the importance of cost guidelines to augment those 
offered in the earlier instructional development, "A Procedural and 
Cost Analysis Study of Media in Instructional Systems Development," 
the Coordination Unit reports, in the final portion of this chapter, 
development costs estimated by each specialist team. 




14 




REPORT: UNIVERSITY OF COLORADO 



BOULDER, COLORADO 



INSTRUCTIONAL DEVELOPMENT TEAM 
Dr. Robert de Kieffer, Team Leader 
Dr. Keith Davis, Evaluation Specialist 
Dr. Donald Darnell, Instructional Specialist 
Dr. Louis Brown, Media Specialist 



INSTRUCTIONAL DEVELOPMENT ACTIVITIES REPORT 



UNIVERSITY OF COLORADO 



Background 

At the beginning of the sub-contract negotiations, it was 
hoped that all institutions involved in the Instructional Systems 
Development Project would attempt to concentrate their efforts and 
energies In two curricular areas; namely. Speech and Business. 

As a result, immediate contact was made with the Department of 
Speech and Drama and the School of Business. Both of these areas 
were Interested in becoming involved In the project at the 
University of Colorado. 

The courses selected for study and analysis were (a) the 
Freshman English taught by the Speech faculty, (b) Business 
Statistics and, (c) Business Management. A course in Business 
Communications was investigated, but due to a resignation by 
faculty that occured right after our initial discussion, this 
endeavor was dropped. 

(a) The Freshman English course at this University consists of 
approximately a hundred sections. Four of these sections were 
taught by the faculty from the Speech and Drama Department. There 
existed a concern, particularly for the said four sections, to 
improve the course. With the approval of the Department Chairman, 
the ISD model was applied by the University of Colorado Project 
Team. 

(b) The Dean of the School of Business discussed the project 
with the University of Colorado Team. The Statistics course 
was deemed to present inherent problems as to the nature of the 
content. With the agreement of one faculty member concerned, 
the team applied the model to this course. 

(c) The interest in the School of Business spread to the faculty, 
particularly one man, of the Department of Production and Manpower. 
With this expressed interest, exploratory sessions were held with 
the faculty and chairman of this Department. 

Plan of Action 



In the case of the English course, the plan of operation 
evolved so the team worked with one faculty member (who happened 
to be on the team). He in turn served as liaison with the other 
three members of the group. The model was followed in the baSiC 
format developed at M. S. U. This model was modified as the result 
of this endeavor and noted in the following paragraph. 
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For the Statistics course, the team contact worked primarily 
with one member of the faculty that taught this course. The 
liaison between the team and other faculty members has been 
incidental. The modified model has been employed with this 
course. 

With the Management course, the team has worked with four 
of the five faculty members teaching this Course, including the 
Department Chairman. The modified model has been followed. 

Specialists 

The team at the University of Colorado has consisted of four 
members, i.e.. Dr. Robert de Kieffer, Director; Dr. Louis Brown, 
Coordinator and Media Specialist; at present Dr. Keith Davis, 
Evaluation; and Dr. Donald Darnell, Learning. 

The University of Colorado team has consistently operated 
a group in each case. These have been of great value because each 
member makes contributions from his own background regardless of 
his assigned role. Both de Kieffer and Brown have extensive 
background in curriculum organization and therefore have made 
helpful suggestions in the area of teaching strategies. 

The Learning Specialist has stated that a person with 
a broader background in learning theory and education^ 
psychology might better round out the team. 

Sequence and Outcomes 

(a) work with the English course was our first endeavor. 

The Michigan State University model and the University of 
Colorado modification that evolved were applied step by step 
with recycling occurring as the need arose. 

Several sessions were devoted to the identification of 
objectives. The approach was to concentrate on student behavioral 
objectives and Robert F. Mager’s book, ‘'Preparing Objectives 
for Programmed Instruction” became a helpful guideline along 
with Benjamin S. Bloom's, "Taxonomy of Educational Objectives." 

Since this course had objectives in terms of the student 
ability to make an oral presentation, two basic types of media 
were employed. 

1. Overhead transparencies to illustrate communications 
concept and outlines of classroom presentations. 
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2. Video tape and 8mm film were employed to observe 

student presentations and for evaluation by students 
as well as faculty. Due to the limitations of 8mm 
equipment, the video tapes were better accepted and 
of course allowed for immediate playback. A planned 
combination of the two media would perhaps have been 
a more desirable approach. 

The activity of the team with the tSD model led to complete 
revision of the English course as presented by the Speech Depart- 
ment. This dour^e^ while no longer being taught in the English 
Department, is now being offered as the basic Speech course in 
the Department of Speech and Drama. 

(b) The Statistics course was the second course to be phased 
into the project. 

The revised model was followed step by step. When we arrived 
at the point of developing teaching examples, it became clear that 
there were not good textbooks or reference available which were 
consistent with the stated objectives of the course. The decision 
was made at this point to develop a basic manual for this course. 

Two different approaches in the preparation of this manual were 
made. One was written in a more traditional style and the other 
was developed along the lines of a programmed text. These two 
manuals will be compared for effectiveness with students. 

Several transparencies were developed for classroom presen- 
tation. These have yet to be evaluated. 

One of the most significant developments with this course 
was a concerted attack on determining the background and achieve- 
ment of students. StudentsP background data is being correlated 
with test results in the course. It is hoped that as a result 
of this phase of the study a more realistic prognosis can be made 
of the probable success of individual students prior to enrollment 
in the course. Coupled with the faculty evaluation, a laboratory 
session is being designed to provide supervised learning experiences 
for those students who have difficulty with the content as deter- 
mined by classroom and/or study assignments. 

Classroom presentations have yet to be resolved. 

(c) The Management course has been the last course to be involved 
in the project. As noted above, the team is working with four 
faculty members who teach this course. More time was thus required 
to arrive at a concensus concerning objectives, but at this writing 
there is apparently a greater understanding of the objectives by 
each of the faculty members. 



This group is at the stage of developing teaching strategies. 
The approach currently contemplated is to consider the student 
perception of the course, and the faculty's . perception in rela- 
tion to the major content areas. Details of classroom presen- 
tations have not been resolved to date. 

In every instance, evaluation of the "change" will he a 
continuous process. 

The Team 



The University of Colorado Development Team has found the 
better procedure. is to have the team of specialists meet as a 
group with faculty members who become a part of the larger team, 
and to work together as a team for the sole purpose of the 
improvement of instruction. The team involvement is needed at 
all policy or decision-making sessions, but the follow-through, 
such as the production or the testing, is decided upon the respon- 
sibility of the appropriate specialist. 

The role of individual members of the team of specialists, 
including the faculty participants, should not be finite. The 
competencies of each member will "come to the top" of the dis- 
cussions as the need arises. This, in no way, implies that 
competencies for evaluation techniques, curriculum organization 
and learning principles and the media field are not needed. 

They are just not emphasized and distinguished as such by the 
team members. This has encouraged our team members to develop 
knowledge and confidence in each other. This approach has also 
built confidence on the part of faculty participants since they 
do not identify individual gains to be anticipated by a particular 
specialty, i.e., "empire building." 

This team approach keeps each member abreast of decisions and 
outcomes. Thus, the time needed to "fill In" or "up-date" a 
member is eliminated. Variations of possibilities and expres- 
sions of personnel opinion need not be gone over several times 
because of an absent colleague. The dangers of receiving 
misconstrued information about the activities of the team, 
including faculty members, is greatly reduced by this face- 
to-face confrontation of all the people involved. 

Common knowledge tends to increase rapport and to build 
confidence. 

This team approach has evolved a variation not previously 
encountered with the other institutions in the overall project. 
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The development of stipategies by the team for pursuing the 
model with a given faculty has proven most vorthwhile. The 
Colorado team has found a need for the team to discuss among 
its members the acuity personalities involved; the problems 
within the courses and ways to solve them in light of these 
personalities and the history of the courses. 

These "staff" sessions not only provide direction for 
the team members but provide for deeper consideration of the 
course content, strategies, and learning problems. It is 
often difficult or detrimental to freely discuss problems 
with the faculty present, particularly if such discussionp 
pertain to the teaching techniques or mind set. 

The Model 



The Colorado team determined early in its involvement with 
the H»S.U* model that the model in its original term contained 
too many operations or steps for practical purposes. As a con- 
sequence, the Colorado modified model (see Figure 3 , p. 21) 
was developed and applied. The modified Colorado model is 
the result of combining several steps in the original 
Michigan State model. 

The modified (simpler) model was used to give direction for 
faculty members. After exposure to this organized approach, 
the faculty members knew What was involved and became more 
cooperative. (VJith a formal Educational Development Program, 
this direction is implied and the exposure of a model of 
procedure would be less needed.) 

It has been our experience that faculty members do not, 
or are reluctant, to respond to a lock-step fashion. The 
nature of the steps in the Michigan State model overlap to 
such a degree that it is inconvenient, even detrimental, to 
try to isolate them in practice. We have found the broader 
categories of the modified Colorado model allow enough flexi- 
bility to be usable and at the same time it retains a syste- 
matic approach. 

The development of teaching examples per se has a tendency 
to be extraneous. This step appears to be more attainable 
when the examples are actually developed as a natural outcome 
of arriving at an organizational and teaching strategy. The 
Colorado model might thus be reduced in steps so that examples 
are included within the strategy. 
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Figure 3. 



Revised Instructional System Development Model 
Employed by the University of Colorado Specialist Team 




R !• Determine Inputs 
E 2, Defining Operational Objectives 
C 3, ?lan Overall Strategy 
Y 4, Develop Teaching Examples 

C 5* Develop Behavorlal Exams 

L 6. Locate, Select and Produce Materials 

E 7, Dry Run 

8. Wet Run 
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It should be noted that the Colorado model attempts to 
combine steps for practical application. We have not attemp- 
ted to eliminate steps for the team of specialists as such, 
but rather to incorporate separate steps into broader cate- 
gories for faculty members. In this respect, the Colorado 
model is the Michigan State model with only broader headings 
identified. The definition of these broad headings are the 
more detailed steps of the Michigan State model. In a sentence, 
the Michigan State model provided the background for the team. 
The Colorado modification provides the systematic arrangement 
of areas of concern by which a larger team of specialists and 
faculty can by guided. The ’‘dry run*' step in both models has 
yet to be tested as a necessary part of the model. We have 
simply not found the time to pursue this step adequately. 

It may have limited utility. 

Reactions to ISDP at the University of Colorado 

1. **Who is going to receive the reports of our dis- 
cussions? The Dean?" (Typical of the concern of 
the faculty. One professor was offered the audio 
tapes being recorded for his keeping before his 
fear was subdued.) 

2. "I was damn mad after the first couple of sessions 
of trying to explain objectives that I could have... 
Then my conscience troubled me so that I could not 
explain the objectives, so I decided to pursue the 
investigation further. I am glad that I didn't give 
in." (Several reactions of this variety were expressed 
after several weeks of meetings. The comment refers 

to the team's persistence in attempting to identify 
course objectives in behavioral terms.) 

3. "I expected to have media pushed upon me." 

(When the application of media equipment was not 
discussed early in the process, this type of comment 
was expressed.) 

4. "I would like to be able to Is there 

some way I can achieve this objective. " 

"Don't we have a — (media) to illustrate 

?" (After objectives had been established 

and queries about evaluation procedure was approached, 
the above references were expressed.) 
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5. "I have worked more hours on this project than 
I do at nty regular job I'* 

6. "If I had a better selection of students and the 
facilities, my course would be great." (Attitude 
encountered from about half of the faculty involved.) 

7. "This project has created more interest and enthusiasm 
for this course than has ever existed." (Pretty 
consistent remark after an introduction of a new 
approach to instruction.) 

8. "I have never seen students so interested and 
attentive in this course!" (Viewing a student 
role-playing situation on video tape.) 

9. "Kids stopped after class consistently to discuss 
their problems as the result of this technique!" 

(Class critique of student performance on film 
and tape.) 

10. **We (faculty) need more time to develop materials 
for our course." (This lament was expressed in 
two out of three cases.) 

11. **We (faculty) have discussed this to the point 
of redundancy. Let's move on to the next step." 
(Believed to be related directly to the awareness 

of the total system and thus a felt need to proceed.) 
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REPORT: 



SYRACUSE UNIVERSITY 



SYRACUSE, NEV7 YORK 
INSTRUCTIONAL DEVELOPMENT TEAM 



Dr« Eugene K, Oxhandler, Team Leader 
Dr» Kenneth Fishell, Evaluation Specialist 
Mr* Milton 1« Patrle, Instructional Specialist 
Mr* Thomas £• Burford, Media Specialist 
Miss Joan I* Novak, Media Specialist 



INTRODUCTION 



Syracuse University shares the concern of Michigan State 
in regard to growing student enrollments and the dearth of 
professorial talent. It too has sought new ways of meeting 
instructional obligations through the employment of th^ newer 
media of communications. 

The following pages report Syracuse Univ'srsity* s part in 
the Investigations of guidelines for analysis of course in- 
struction and implementation of the newer media. In this study 
an instructional specialist, or a media specialist, examines 
the Instructor's resources and class limitations, determines his 
course objectives, and develops media and tests that fit the 
instructor's needs. These materials are tested and evaluated 
to determine their appropriateness. These procedures are 
repeated to further refine objectives and further modify the 
materials. 

It is appropriate here to pay tribute to the excellent 
leadership provided the Syracuse instructional development team 
by the recently deceased Dr. Eugene K. Oxhandler. His depth of 
experience in researching the field of newer media and instruct- 
ional development was critical in much of the effective work 
in this portion of the project investigation. 



25 



INSTRUCTIONAL DEVELOPMENT ACTIVITIES REPORT 



SYRACUSE UNIVERSHY 



In keeping with the parameters established by the project 
designers, two courses were chosen. Selection was based upon 
three criteria: 

1. The course was o£ a basic nature in relation to the 
total program* 

2. The course served a large student population (500 students 
per semester). 

3. The willingness of the instructor to submit to the 
rigorous demands of the systems approach to the develop- 
ment of his course* 

Based on these criteria, the courses chosen were Psych 5, 
"Foundations of Human Behavior," and CFEd. 200, "Introduction 
to Philosophy of Education." 

Psych 5; The General Structure of the Course 



Description: 

Introduction to the fundamental principles of mental 
life and human behavior. Significance of psychology 
in human relationships and self-understanding. Not 
open to freshmen. 

The course was divided into six sections. Each section ranged 
from 150-250 students, including one small honors section of 25-30 
students, giving a total population for the course of about 700 
students per semester. Generally, Psych 5 is a one-semester 
course which is repeated every semester. 

The Initial Teaching-Learning Situation 

Since our efforts centered around Dr. Joseph Sturr, of the 
Psychology Department in the School of Liberal Arts, we will attempt 
to describe only the initial teaching- learning situation in which 
he was involved. Initially, Dr. Sturr* s class met 3 times a week 
for 1 hr. The class consisted of approximately 180 sophomore 
students. This class size dictated that it must be held in a 
large lecture hall. As a result, the initial classroom was a 
large barnlike room, of poor acoustical design and quality, x^ith 



r 



a capacity of 200/. Initially, there were no facilities for media 
usage. The only facility available to the instructor was a 5 x 12 
ft. blackboard, mounted on the right of the front wall. The poor 
acoustical quality of the room and the lack of adequate amplifica- 
tion facilities made lecturing most difficult, if not impossible. 

All too often the lecture could not be heard or understood. Although 
the instructor resorted to frequent use of the blackboard, informa- 
tion placed upon it could not be read by anyone at the reax of the 
room. At our request the audio-visual service department of the Center 
for Instructional Communications, placed an overhead projector and 
a large tilting screen in the room. This helped improve the situa- 
tion. Although we made many attempts to make the best of the 
situation, it soon became apparent that this room would not lend 
itself to effective use of the various media available. With this 
in mind we decided to attempt the scheduling of Psych 5 for the 
multi-media classroom facility in the Newhouse Communications Center. 
Actual scheduling was accomplished for the second semester (*65 * ^66). 

Initial Contact With the Instructor 

Initial contact with Psych 5 had been established through Dr. 
Joseph Sturr, one of the five instructors teaching the course. Dr. 
Sturr had previously visited the Center for Instructional Communica- 
tions, seeking the development of materials for his course. Because 
of his purpose and his eagerness when confronted with some new media, 
we approached Dr. Sturr to determine his willingness to participate 
in the media- oriented approach to course development which our Project 
Model offered. He was ready and willing to assist us in investigating 
this program. 

The Model, as is, first calls for a rigorous definition of be- 
havioral objectives. Dr. Sturr admitted that he had had no formal 
training in methods of instruction. In addition, both Dr. Sturr and 
the media specialists lacked desired time and back up personnel to 
permit the called-for entry to the Project ISD Model. However, Dr. 
Sturr freely discussed his course outlines, notes, readings and general 
instructional aims, permitting us to enter the model at points involv- 
ing decision-making for the production of materials. This will be 
further discussed under .initial strategies. 

Originally, we had intended to gradually involve all five in- 
structors in Psych 5 in the development of the course. A campus 
survey of students* opinions about existing courses indicated that 
Psych 5 was considered dull, uninteresting, and difficult to digest. 
However, as planning sessions evolved, certain uncontrollable factors 
soon became apparent. The five instructors had three distinct and 
different ideas about course content and how it should be taught. 

This was primarily due to their divergent backgrounds in psychology. 
Two instructors considered themselves physiological psychologists, two 
more described themselves as clinical psychologists, and the fifth 
was a psycho- linguistics specialist. These unresolved instructional 
biases, complicating the production of specific objectivtiS, could not 

27 



o 



be considered as a ground from which we might survey media and 
course development. Therefore, in order to proceed, with limited 
time and resources, we elected to work primarily with Dr. Sturr. 

CFEd. 200; Introduction to Philosophy of Education 

Description: 

An examination of philosophic issues which bear upon 
education. A comparative treatment of conflicting 
beliefs about reality, knowledge, and value Is made 
in terms of idealism, realism, and pragmatism. Pre* 
requisite: none, although prior work such as Philosophy 

131 6 (Amer. Phil.) is highly desirable. 

The General Structure of the Course 



CFEd. 200 is a one-semester course repeated each semester. 
Initially, the course met Thursday evenings for two hours each 
evening. A student population of about 180 had enrolled for the 
first semester ('65 - "66). About 85% of this population was 
comprised of practicing teachers enrolled for graduate work. The 
instructor was Dr. Thomas Green of the Cultural Foundations Depart- 
ment in the School of Education. 

Initial Teaching-Learning Situation 

The class size (100 / graduates), again, dictated the use of 
a large lecture hall. The initial lecture hall used by Dr. Green 
was similar in limitations to the one used by Dr. Sturr for Psych 5. 
Effective use of several media was limited. Again, we were success- 
ful in scheduling C.F. Ed. 200 into the multi-media classroom facilities 
of the Newhouse Communications Center for the second semester ('65- *66). 

Initial Contact with CFEd Instructor 



Anticipating the inception of the ISD Project at Syracuse Univ- 
ersity, Dr. Oxhandler personally contacted Dr. Green in an attempt to 
determine his Interest in participating in the project. His decision 
to contact Dr. Green was prompted by two beliefs. First, through his 
professional contacts with him, he believed that Dr. Green was keenly 
Interested in experimenting with instructional techniques. Second, he 
believed that the nature of the course, "Introduction to the Philosophy 
of Education," presented a unique challenge to the ISD demonstration 
team in the application of the model and In the design and development 
of appropriate media. A preliminary conference with Dr. Green revealed 
that he was ready and willing to participate in the project. We be- 
lieve that one of the major reasons Dr. Green chose to participate in 
the project was the anticipation of using the responder-system in the 
multi-media classroom, to gather data for measurement and evaluation 
of his Instructional techniques and their influence on the decision- 
making process and opinions within the students. 
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In the initial planning sessions we attempted to induce 
Dr, Green to define his behavioral objectives. Dr. Green* s 
reaction to this approach was adamant. He immediately asserted 
that he did not believe in stating behavioral objectives, since 
they were verbally apparent in his lectures. Therefore, in order 

TJO ^ 4 41% 4* 4 ««A M 

and sought other entry points into the proposed project model. 



Initial strategies designed and adapted for facilitating 
entry into the model will be discussed in detail in succeeding 
sections of our final report. 



HEURISTICS OF INSTRUCTIONAL SYSTEMS DEVELOPMENT ACTIVITIES 

The following portion of the final report is a relation of 
the heuristics of instructional systems development activities, 
as they evolved at Syracuse University. We have attempted to 
express these heuristics as they relate to the application and 
implementation of the hypothetical ISD model. These heuristics 
are suggested solutions to the problems that arose during our 
activities in course development. 

PREPARATION FOR INSTRUCTIONAL SYSTEMS DEVELOPMENT ACTIVITIES 

Upon receipt of the sub-contract, the demonstration team 
leader, Dr. Eugene K, Oxhandler, selected an evaluation specialist 
and two media specialists. This specialist team then analyzed the 
ISD model in terms of the roles and functions of the team members. 
Who was going to do what? It was apparent that we had to determine 
the "What”. Therefore, we analyzed the specifications written into 
the project, looked at the model again, and reduced the "what** in 
the model to a set of basic operational objectives. Underlying 
these objectives was the assumption that these are the kinds of 
functions and activities that a development team must pursue in 
order to facilitate the selection, production, and implementation 
of media in course development. This concept is, of course, pre- 
dicated on the initial decision that our approach to course devel- 
opment activities was to be media-oriented. This set of operational 
objectives may be expressed as follows: 

The basic objectives of the development team 
will be to assist the instructor when requested, 
in ■ 



1-- specifying the general educational goals 
for the course 



2- - specifying his specific objectives for 

the course in terms of the exiting or 
terminal behaviors of the learner 

3— specifying the students' entry behavior 
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4- -specifying content 

5- -devel oping rationale for pre and post 

exans 

6 — developing evaluation instruments with 
student data as well as with media 
information 

7- -combining and analyzing total input 

data, (i.e., data gathered via 
activities 1-6 above) 

8- -planning overall strategies 

9— developing teaching examples of 
determined content 

10— choosing representative informational 
forms, (i.e,, developing information 
models based on perception and 
learning theories) 

11— selecting appropriate transmission 
vehicles, (various media) 

12- -developing the instructor’s skills 
in the methodology of various media, 

(both hardware and associated soft- ware) 

13— evaluating and refining the instruc- 
tional packages produced. 

It should be noted here that the phrases "to assist” and 
*'when requested" are the keystones in implementing these opera- 
tional objectives at Syracuse, We were and are not in a position 
to *' force” the instructor to accept these services, Oa the 
other hand, we were and are obligated to bring to bear every 
possible influence available which might elicit a ”felt need ” 
that will lead the instructor co request such services. In some 
instances the instructor could accomplish many of the functions 
implied in these operational objectives without resorting to the 
services of the development teams 

Cnee these operational objectives had been specified, the 
team members’ responsibilities were delineated, the role and 
function of each team member was originally established as follows: 

-—the evaluation specialist would assist the 
instructor in ^’^^omplishing objectives 
1, 2, 3, 4, 5, 7, 8, and 13 

--the media specialists would assist the 
Instructor in accomplishing objectives 
7, 8, 9, 10, 11, 12, and 13 
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Note that both the evaluation specialist and the media 
specialist have a mutual responsibility: to assist the 

instructor in accomplishing objectives 7, 8, and 13. These 
specialists may work together, in aiding the Instructor and 
accomplishing a specific objective, or independently of each 
other. Delineation of role and function satisfied the ne'.ed 
to know "who was going to do what?" 

"What courses shall we involve?" was the next question that 
had to be answered. The answer to this question was, in part, 
predetermined by the course criteria set by the project designers. 

We believe an equally important factor hinged on identifying 
faculty members who demonstrated an interest in media and/or 
course development. We attempted to meet all of the predetermined 
criteria for course select ion j as specified in the project 
agreement. However, we placed precedence on the instructor's 
demonstrated interest in media and course development activities. 

The events that led to the ultimate selection of the instructors 
were described in the section of this report dealing with the 
background of course development problems. .. 

Actual course development activities commenced when initial 
contact was established between the course instructors and the 
development team. This initial meeting was scheduled by the 
team leader. It was designed to establish a working relationship 
between the team of specialist and the instructor. During this 
meeting the instructor was introduced to the members of the team 
and appraised of the role and function of each specialist. At 
this meeting preliminary plannl.ng sessions were scheduled. The 
demonstration team decided to initiate the data- gathering process 
during these preliminary planaiag sessions. 

GATHEPaWG INPvIT DATA 

The first preliminary- planning session was devoted to 
collecting basic input data concerning the nature and scope 
of the courses, including; 

---course objectives 

--general objectives 

— -number of students 

-—available finances 

---materials available 

---classroom facilities 

This preliminary session progressed well and we were able to 
delineate some general objectives for the courses. The data 
gathered at these preliminary planning sessions provided a basis 
for formulating initial strategies for course development activities. 
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INITIAL STRATEGY 



Initial Strategy for Course Development Activitlesi PSYCH 5 

Originally, the development team decided that developmental 
activities fer this caurse would grow, primarily, out of course 
development planning sessions* These prior planning sessions 
were designed to be directive in nature* During these sessions 
the development team would assist the instructor in moving through 
each phase of the development sequence as reflected in the hypo* 
thetical model* However, it soon became apparent that this was 
a very time-consuming process* The instructor involved had net 
been provided with release time to pursue these activities, and 
he was carrying a full instructional load* This, of course, made 
it impossible for him to devote the necessary time to pursue this 
approach tc course development. In addition the instructor was 
convinced that developmental activities could proceed without 
detailed statements of specific objectives for content and students’ 
entry and terminal student behaviors. He was, however, quite 
willing to allow the development team to seek out these objectives, 
as long as the search did not place excessive demands on his time. 
The instructor's time was a critical factor in our consideration 
of new strategies in implementing developmental activities. 

With this in mind the developmental team sought other entry 
points into the model. Though Psych 5 was part of a continuing 
instructional program, the development team decided that the 
instruction in this ongoing course would reflect the instructor's 
current philosophy regarding the content and objectives of the 
program. In addition it was decided that the ongoing instruction 
would also reflect "teaching examples of determined content" and 
"representative informational forms". These concepts provided us 
with new entry points into the model. 

Strategies Devise d to Facilitate Course Development Activities 
for CFEd . 200 

The approach to CFEd. 200 deviated from the set pattern 
implied in the hypothetical ISD model. Permission to deviate from 
the prescribed procedure had been pre-arranged by Dr. Oxhandler 
with Dr. John Barson. The initial approach was observation- oriented. 
This decision was based on prior experience in developing media 
for a course in statistics. The development activities in this 
situation were based on class observations. The resultant media 
generated proved to be both appropriate and useful. Therefore, 
we believed the "observation-verification" technique would faci- 
litate implementation of the model, particularly in a situation 
in which the instructor is not provided with released time to 
pursue development activities. This belief has been confirmed in 
the light of our experiences with facilitating course development 
through selecting, adapting, and designing appropriate media. 

This observation-oriented strategy involved two interdependent, 
concurrent phases. The first phase was to observe every lecture 
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of the ongoing instruction and to analyze the associated materials 
and data generated by the Instructor and his students. These 
observations and analyses were made by the media specialists. The 
observations were designed to identify and record evidence of: 
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behavioral objectives 

....teaching examples of determined content 

——representative Informational forms 

instructor*s repertoire of teaching methods 

. — -areas where various media would facilitate 
instruction 

——possible transmission vehicles 

The second phase of this observation-oriented approach 
involved presenting the observations and analyses to the instructor. 
This was accomplisyed during weekly discussion-planning sessions. 
The purpose of this phase was to validate our observations. 3)id 
they accurately reflect the instructor's objectives in terms of 
content and the desired terminal behavior of the student? If they 
did, they could be used as a basis for decisions concerning 
"transmissions vehicles" and recommendations for the ultimate 
"collection, design, and production of specific media". If 
they did not, the ensuing discussion usually shed light on what 
the instructor had originally intended. The flow of events in this 
revised approach are depicted in Figure 4, on page 34. 



Figure 4 



Revised Instructional System Development Model 
Employed by Syracuse University Specialist Team 



INSTRUCTIONAL SYSTEMS DEVELOPMENT MODEL 



INITIAL CONTACT 

I 

i 



COLLECTION OF BEHAVIORAL DATA - 







j 

ANALYSIS OF DATA^— 

f 

1 





..J 

•Jr ; 

o 

Q 

IH 


1 


I 

'/ 


NO PRIMARY CONCERN FOR MEDIA 


i 

1 

\ 


a 

D 


ACTUAL INST. 


ACTUAL 


PRE-TEST 




BEHAVIOR 


STRATEGIES 


CONTENT 


X 

M ! 








\ 

! 


z 

o 


COMPARE WITH 


COMPARE WITH 


1 

\/ 

COMPARE WITH 


1 


1 

\ 1 
[ 1 


DESIRED 

BEHAVIOR 


^ DESIRED 
STRATEGIES 


DESIRED TEST 
BEHAVIOR 











\ 





COMBINED INST. & 
MEDIA STRATEGIES 



V 

RESTRUCTURED COURSE 




COLLECT NEW 
DATA 



34 



This observation-verification method proved fruitful and 
provided the basis of all developmental activities that have 
generated media packages to date. 

New Strategy Devised to Facilitate Course Development Activities 
for PSYCH 5 



Based on our experiences with the initial strategy devised 
to facilitate course development activities for PSYCH 5, as 
outlined above, we soon "felt the need" to change strategies. 

By the end of October, we had made very little^ progress 
in attempting to follow the model "as is." At this point the 
decision was made to shift our strategy towards the "observa- 
tion-verification" approach as specified for the CFEd. 200 
course . 

As time progressed, visable progress was noted in the form 
of appropriate media generated. As a result we soon adopted 
the same "observation-verification" strategy for both courses. 

COMPETENCIES OF SPECIALISTS INVOLVED 

The following specialist personnel has been involved in 
the ISD Project^ 

1. Dr. Eugene K. Oxhand ler 

2. Dr. Kenneth Fi shell 

3. Mr. Cyril Koch 

4. Mr. Milton Patrie 

5. Mr. Thomas E. Bur ford 

6. Miss Joan I. Novak 

7 . Mrs . Diane Oxhandler 

8. Mr. Charles Bidwell 

9. Mr. Richard Muller 
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Personnel; Backgrounds and Major Contributions to Project 
1. EUGENE K. CXHANDLER 



Capacity : Director 

Academic Training : B.S. in Ed. -Art Ed., Teachers College, 

Columbia University, 1944; M.A., Fine and Industrial Art, 
Teachers College, Columbia, 1945; Ed.D,, Art and Audiovisual 
Education, same, 1954. 

Teaching Experience ; Associate Director, Center for 
Instructional Communications, Research, Teaching, 

Advisement, Administration, Syracuse University, 1960; 
Assistant Professor, Research, Training, Advisement, 
Pennsylvania State University, 1957-1960; Associate 
Professor, Teaching, Advisement, Administration Head- 
Art Department, Director of AV Education, Director 
Film Libraries, State Teachers College, East Stroudsburg, 
Pennsylvania, 1953- '57; Teaching of Art and Industrial 
Art, Advisement, Administration, Director of AV Educa- 
tion, Manager of Cooperative Film Library, State Univ- 
ersity College of Education, Oneonta, N. Y., 1948- '53. 

Publications ; A Computer Simulation of a Statewide 
Film Library Network: A Feasibility Study for Actual 

Operation , Syracuse, New York, Final Report USOE Project 
No. 5-0272, Contract No. OE-5-16-024, Sept. 1966 with 
Fred Christen, 47 pp.; The Three "R's" Plus O.R. — 

A Systems Approach to Educational Problems , with 
Eleanor Taylor, USOE Contract No. OE-3-12-4125 Syracuse, 

New York, 1964, 186 pp.; The Findings of Research on the 
Newer Media of Instruction , Council for Administrative 
Leadership, N. Y. State Education Department, June, 1962; 

New Systems for Education Suggested by Operations 
Research , Communications '’Convocation Review," N. Y. 

State Education Department, January, 1965; Bringing 
the "Don's" Up-to-Date — A New Approach to an Old 
Educational System , AUDIOVISUAL INSTRUCTION, October, 

1963, 5 pp.; New Trends in AV Teacher Education , with 
G. M. Torkelson, AUDIOVISUAL GUIDE AND EDUCATIONAL 
SCREEN, December, 1960; Can Subliminal Stimuli Teach ? 
AUDIOVISUAL COMMUNICATIONS REVIEW, May- June, 1960; 

A Report on Six Research Studies , AUDIOVISUAL COMMUNI- 
CATIONS REVIEW, May- June, 1960, and others. 
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Professional Organizations ; NBA; Department of AV 
Instruction; Association for Higher Education; 

American Educational Research Association; Kappd 
Delta Pi; PDK, American Association of University 
Professors; International Council for Research on 
visual Information, Milan, Italy; Educational 
Research Association of New York. 

Contributions to Project ; Provides leadership 
and direction to the demonstration team. He advises 
the principal researcher in matters of administrative 
and financial decision-making. 

KENl;.ii:H Ni FISHELL 

Capacity ; Evaluation Specialist 

> 

Academic Training ; B.S., Education M., Education D. 
in Curriculum, 1964, University of Rochester. 

Teaching Experience ; Assistant Professor, University 
of Rochester, 1961-1965; Research Assistant, University 
of Rochester, 1959-1961; Principal, Rush-Henrietta 
Schools, Henrietta, New York, 1956-1959; Teacher, 
Rush-Henrietta Schools, 1955-1956; Teacher, Newark 
Public Schools, 1952-1955. 

Research Experience ; Coordinator of Educational Media 
Laboratory for Inter-University Project I (Ford Foundation 
Experimental Project in Teacher Education) . Consultant 
with several Rochester area schools on research in pro- 
grammed instruction. Consultant with New York State 
Demonstration Project in Mental Retardation. Consultant 
with local Rochester industries on research. 

Publications ; Author of articles on research in mental 
retardation and programmed instruction. 

Professional Organizations ; PDK, AAUP, AERA, NEA, 

DAVI, NSSE, NOME, AAAS, ASCD, and local and state 
branches of these organizations. 

Mai or Contributions to Project ; Advises the demon- 
stration team in matters of design of evaluation instru- 
ments and the collection of evaluation data. 



3. CYRIL M. KOCH 



Capacity : Research Assistant 
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M.A., University of Michigan, 1962; Ph.D., Syracuse 
University, 1965. 



Teaching Experience : Course Programmer, Center for 

Instructional Communications, Syracuse University; 
Elementary School Teacher, Ardsley, New York; Director 
of Lighting and Sets for TV Production, University of 
Michigan; Research and Production Assistant on two 
series of Regents Educational TV programs, WPIX Channel 
11, New York. 



Research Experience : Co-author and producer of Say 

What You Mean , feature presentation of the. Demonstration 
Center, Hall of Education, New York World's Fall ; 
Currently studying the comparative effectiveness among 
high and low IQ students of a short TV series and 
two-sound- films trip versions of that series. 

Professional Organizations : DAVI 

Major Contributions to Project : Mr. Koch was the princi- 

ple researcher of demonstration team (*65-'66). He 
developed and implemented initial strategies in the 
application of the ISD model. 

4. MILTON I. PATRIE 



Capacity : Research Assistant 

Academic Training : B.S., SUNY College, Buffalo, New York, 

1951; M.S., Indiana University, 1952; NDEA Educational 
Media Institute at Syracuse University, Summer, 1965; 
currently engaged in doctoral study at Syracuse University. 

Teaching Experience : Lecturer, Educational Media Insti- 

tute, Syracuse University, Summer, 1966; Audiovisual 
Instructor, Greenwich (Connecticut) Public Schools, 
1961-'65; Instructor, Central Connecticut State College, 
Suuuuer , 1960; Assistant Professor, University of Maine, 
1957- '59; Assistant Professor, Western Illinois University, 
1956- '57; Elementary Teacher, Los Alamos (New Mexico), 
Public Schools, 1954- '56. 
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Publications ; Author of articles on preparation of 
visual materials and viewing standards for projection; 
contributor to research abstracts. 

Research Experience ; Research Assistant, Instructional 
Systems Development: A Demonstration and Evaluation 

Project, 1965- ‘66; consultant with Niskayuna Public 
Schools on Individualized Instruction, 1966. 

Professional Organizations : Phi Delta Kappa, DAVI, 

NYSAVC. 

Maj or Contributions to Project : Mr. Patrie was respon- 

sible for design and production of media developed for 
courses ('65- *66). Presently acting as coordinator of 
project developmental activities. 

5. THOMAS E. BURFORD 

Capacity : Research Assistant 

Academic Training : B.A., SUNY, Buffalo, 1957; Ed.M., 

S.UNY, Buffalo, 1964; NDEA, Educational Media Institute 
at Syracuse University, Summer, 1965; Graduate of 
Empire State Military Academy, Officers Candidate 
School, Commissioned Officer USAR & NYARNG, 1958; 
currently a Ph.D. candidate in Educational Communi- 
cations Media, Syracuse University. 

Teaching Experience : Teacher--AV Coordinator, Depew 

School System, Depew, New York, 1^57- '58; Teacher — AV 
Coordinator, Lewiston-Porter Central School, Youngs- 
town, New York, 1958- '60; Director of AV Education 
and Teacher, Lewiston-Porter Central School System, 
Youngstown, New York, 1960- '66; Instructor, ESMA- 
OCS, Branch School, Syiacuse, New York, 1966. 

Research Experience : Background in research methods 

through graduate work; i.e., courses in The Experiment 
in Education, Design, and Evaluation, Problems in 
Educational Research (Research Readings). 

Professional Organizations : PDK, DAVI, NYSAVC, 

NYSTA, and local branches of these organizations. 
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Major Contr:&butions to Project ; Mr. Bur ford assumed the 
position of principle researcher in the project in Sept- 
ember, 1966. He is now responsible for continued deve- 
lopment and implementation of strategies whic h facilitate 
the application of the ISD model in course development, 
('68- *67) . 

6. JOAN I. NOVAK 

Cap acity : Graduate Assistant 

Academic Training : B.S., SUNY College, Albany, New York, 

1961; MSEd., SUNY, Potsdam, New York, 1965; English 
Institute study at St. Lawrence University, Canton, 

New York, Sunmer, 1962; currently engaged in doctoral 
study at Syracuse University. 

Teaching Experiences : English teacher, Gouverneur 

Junior High School, New York, 1961- *64. 

Professional Background , other : Counselling as welfare 

caseworker, Jefferson Co., New York, 1964- *65. 

Professional Organizations : DAVI 

Maj or Contributions to Project : Miss Novak assisted 

principle researcher in developing and facilitating 
the implementation of course development strategies, 

(1966- '67). 

7. MRS. DIANE OXHANDLER 

Capacity : Dr. Green's Graduate Assistant, 1965. 

Academic Training : B.A., Wilanette University, Salem, 

Oregon, 1957; M.A., Syracuse University, 1963; 

currently e^.gaged in doctoral study at Syracuse University. 

Teaching Experience : English Teacher, Secondary School, 

Oregon, 1959- '61; Substitute Teacher, New York State, 

1963. 

Professional O rganizations : John Dewey Society, and 

Philosophy of Education Society. 

Ma jor Contributions to Project : Mrs, Oxhandler was 

Dr. Green's graduate assistant and, in this capacity, 
was able to contribute valuabel insights into; 
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Dr. Green, "the man," (personality, attitudes, and 
opinions), and course content. This was invaluable 
in formulating strategies we could use when working 
with Dr. Green. 

8 . CHARLES BIDWELL 

Capacity ; Production specialist 

Academic Training : B.A., McMas ter University, 

Hamilton. Teachers College, Ontario, Canada, 1959; 
Permanent Teaching Certificate, same, 1960; B.Ed., 
University of Toronto, 1964; MSEd., Syracuse Univer- 
sity, Syracuse, New York, currently being completed. 

Teaching Experience : Elementary School Instructor, 

three years, Hamilton, Ontario, Canada; Elementary 
School Assistant Principal, two years, Hamilton, 

Ontario. Canada. 

Professional Organisations ; PDK, OPSMTF (Certified 
Teacher, Ontario Public School Men Teachers Federation) . 
DAVI, CAVA (Canadian Audio-Visual Association) . 

Mai or Contributions to Project ; Assisted in the 
design and production of media developed for courses. 

9. RICHARD MULLER 

Capacity ; Computer Programmer 

Academic Training ; B.A., Psychology, Amherst College, 
Massachusetts, 1964; currently engaged in doctoral 
study at Syracuse University, New York. 

Teaching Experience ; Lecture ’ in Media I and Media 
II courses. Department of Education, Center for 
Instructional Communications at Syracuse University, 
during 1964 and 1966; Instructor in Production 
Methods for TV in Summer Institute for Media Specialists 
1965, and in Computer Programming for same, summer 
Institute of 1966. 

Publications ; Articles on Computer Programmer 

Systems for and about CAI. 
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Research Experience : Currently an NDEA Research 

Fellow at Syracuse University 

Professional Organizations : PDK, Alpha Epsilon Rho, 

DAVI, AERA, and ACM. 

Major Contributions to Project : Mr. Muller is present- 

ly developing computer programs for use with the 
responder system-computer hookup in the multi-media 
classroom. The computer programs are being designed 
to provide the instructor and the student with almost 
immediate feedback of a detailed analysis of test results. 



CONCLUSIONS DERIVED FROM INVESTIGATION OF INSTRUCTIONAL 
DEVELOPMENT AND MEDIA IMPLEMENTATION 

Generalizability of the Development Model: ISP Model ”As Is” 

Implementation of course development, through the applica- 
tion of the implied procedural sequence as specified in the 
model "as is,” might best be accomplished under the following 
conditions : 



— instructors are provided sufficient 
released time to pursue developmental 
activities 

— complete prior planning of a new 
course, V7herein ongoing instruction 
could not be observed 

ISD Model With Some Modifications 



The basic problems we encountered, in attempting to fol- 
low the model "as is," stemmed primarily from two basic facts: 

(1) the instructors were not provided 
with released time to pursue develop- 
mental activities as specified in 
the model 

(2) they did not "feel the need" to 
specify behavioral objectives 
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To off-set these two factors, we developed the "observa- 
tion-verification” technique which provided entry into the model 
at several other points. 

The suggested modification, therefore, affects only the 
implied sequential flow of events, not the events themselves. 

In addition, we recommend the inclusion of "Analysis of Instruc- 
tor's Entry Behavior" as one of the events and/or activities 
in the "Gathering of Input Data." This recommendation is based 
on our observations that the instructor, in many instances, 
will need training in the effective utilization of both the 
hardware and software of the media generated. 

Substitute ISD Model 

If we were to propose a substitute ISD model, it would be 
primarily based on "observation-verification-utilization" of 
behavioral data concerning both the instructor and the students^ 
entry and terminal behaviors . We believe the media generated 
from this type of "combined input data" would be more appropri- 
ate to both th^ "Needs of the instructor" and the "Needs of 
the students." 



REPORT: 



SAN FRANCISCO STATE COLLEGE 



SAN FRANCISCO, CALIFORNIA 

INSTRUCTIONAL DEVELOPMENT TEAM 
Dr. A. Daniel Peck, Team Leader 
Mr. Francis X. Moakley, Media Specialist 
Dr, Freeman F. Elzey, Evaluation Specialist 



INSTRUCTIONAL DEVELOPMENT ACTIVITIES REPORT 



San Francisco State College 



Development of Project Team 

The first analysis of the Michigan State Proposal as out- 
lined by the Director, Dr. Barson, required that a project 
team be organized which could provide specific skills. Per- 
sonnel indicated were an administrative head to direct oper- 
ations at San Francisco State College, a qualified media 
specialist, a specialist in evaluation and learning psychology, 
and a specialist in teaching organization and methodology. 
Assigned to the team to meet these qualifications were Dr. A. 
Daniel Peck, Director and Instructional Specialist; Mr 
Francis X. Moakley, Media Specialist; and Mr. Freeman Elzey, 
Evaluation Specialist. 

The team was designed to operate as a group with the 
realization that each member of the team, while well qualified 
in his own special areas, was also qualified in each of the 
other areas to a considerable degree. The first meetings of 
the team considered the parameters and rationale for course 
selection. Prior to the first meetings with instructors, 
the team interviewed students and pursued the College offerings 
for appropriate courses. On the basis of its deliverations the 
team selected the courses reported below. 

The Selection of Courses 

Course selection was based on the following parameters: 

1. The courses which were in the lower division of 
the College. 

2. Courses which enrolled in excess of 250 students 
per semester. 

3. Courses which were basic requirements. 

4. Courses which in the opinion of students and 
faculty should be improved for receptivity and 
effectiveness. 
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A basic dufficulty was encountered at San Francisco State 
College since there are no large sections (250) . To meet the 
250 enrollment parameter, courses which were multiple section 
were examined and particularly those courses which had the 
largest sections. The choice narrowed to a basic Biology course. 
Social Science 30, and United States History. Interviews with 



the Deans of each of these Schools resulted in faculty recom- 
mendations and recommendations for specific sections. The team 
visited each of the sections suggested and interviewed each of 
the instructors. The unanimous selection of the team for the 
first try was Social Science 3C , a general education course 
designed for sophomores and non-Economics majors which treats 
rudimentary elements of economics and economic theory. The 
course was taught by Dr. Arthur Benavie, a distinguished 
mathematical economist and Associate Professor in the Department. 



For the second course to be selected, the History Department 
recommended History of Western Civilization, History 4.1, which 
met all of the requirements previously stated. The instructor 
for the second course was Mr. William Bonds, an Assistant Pro- 
fessor in the History Department, and an eager participant. 



In addition to these courses, the Psychology Department 
came forward with the request that we review and recommend a 
systems development treatment for their basic Educational Psycho- 
logy course, and the Biology Department made a strong request 
that the team be allowed to sic with them in course planning 
sessions . 



Development of Social Science 30 (Economicsl 

The first interviews with Dr. Benavie revolved around the 
behavioral statement of objectives and an introduction to the 
general method of operation of the team. There was a consi- 
derable difficulty incurred in coming to precise behavioral 
definitions and dividing them into operatable sub-units. After 
several meetings, the team considered that more efficient use 
could be made of time if it concentrated on the objectives 
developed to that point and moved to determine some class 
operations. The Evaluation Specialist, the team Director, and 
Dr. Benavie arrived at an ertry and exit measurement device 
which categorized the various levels and kinds of behavior 
outlined in the basic objectives. This instrument was adminis- 
tered half way through the first semester (preparation semester), 
at the beginning of the second semester (operating), and at 
the end of the second semester. Some differences were noted 
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betwsen the j^re" snd post-exeiuinations which indicated that 
the objectives stated were being met to some measurable degree. 
Sophisticated statistics were not applied. 

Certain portions of the Economics course v/ere given 

-J An cinAo c Pn1“.Q S eemed to be having some diffi- 
culty in these areas. Since examinations revealed that students 
knowledge of graphic language and methods was hampering their 
success, special attention was given to remedial measures in 
this area. A program of 195 frames was developed as a remedi- 
ation measure. The tabular methods of the instructor utilized 
the blackboard to a great extent, and an attempt was made to 
substitute an overhead projector as a presentation tool. A 
series of transparencies was developed for this purpose. To 
facilitate easy use of the overhead projector, the room was 
repainted to provide a screen area 8' by 23’ above the black- 
board. Some difficulties were encountered in instructor 
acceptance of the intrusion of the overhead projector between 
the instructor and his class. A series of transparencies was 
also developed to provide a treatment of basic equation tech- 
niques as a remedial measure. Also introduced into the class 
as hand-out materials wertj specific articles written by the 
instructor for purposes of clarification of particular issues. 

At the end of the second semester the instructor and the 
team considered that it might be advantageous to continue to 
assess some of the developments and to make additional try-outs 
and reorganizations of materials. This work has proceeded for 
a third semester and may continue during the final semester of 
the project. 

Development of History 4.1 (History of Western Civilization)_ 

During the second year of the project, the team began 
interviews with Mr. Bonds of the History Department. In the 
initial phases of determining objectives, the same difficulties 
were encountered as had been experienced in the Economics dis- 
cussions. There was a considerable reluctance to come to the 
behavioral statement of objectives. Objectives were stated 
primarily in content and chronological content outline form. 
However, after several tries with the entire team and consid- 
erations of evaluation techniques, some beginning successes 
in a behavioral statement of objectives were apparent. Opera- 
ting upon these and the evaluation base, the Media Specialist 
and the instructor have been developing course materials of a 
slide-filmstrip nature. Continuing evaluation techniques 
were devised which utilized immediate response systems in 



conjunction with overhead presentations. For later consider- 
ation, electronic response techniques were discussed and may 
be employed. As a part of the instructor assessment, closed 
circuit video tape recording of instruction sequences has 
been planned and is proceeding for the first semester. Pre- 
and post-cvaluatioii techniques are being designed by the 
instructor in conjunction with the Evaluation Specialist. 
Individuation technique will include carrel study of tape- 
slide sequences under development. 



Courses Not Selected 



Among the courses not selected were the Biology and 
Psychology offerings . Other courses which the team were asked 
to consider but did not feel that they could undertake at this 
time were English, Humanities, and Art. It is evident that 
the project has created an interest in a systems development 
within these areas. While the team cannot work v;itL the number 
of instructors who have requested assistance, they have contin- 
ued to work with both Biology and Psychology in course planning 
activities . 



San Francisco State College accepted the Instructional Systems 
Development Project as a challenge and an opportunity for instruc- 
tional improvement. "Systems" is a strange work to apply to an 
activity which has always been thought of as highly personal. 

Our first question w^.s whether the system can be applied to 
tailor the model to human characteristics operant on the indi- 
vidual campus. 

San Francisco State College is an aggressive, dynamic, 
highly individualistic campus. Most of our students are actively 
involved in the community making a living- -sometimes changing 
the community, and in the college getting an education as 
efficiently as possible. They are in a hurry and they are eager. 
They are ready to be involved. 

Both faculty and students are concerned about the quality 
and method of instruction. It was important, therefore, to 
interpret the model to reflect rather than oven^helm their 
concerns . 

The courses which we selected to develop were those with 
which both faculty and students were dissatisfied. The courses 
were required, frequentl.y dull, and often not voluntarily 
selected by faculty. 
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The first instructor and course selection was made with 
the cooperation and the recoimnendation of the head of the 
Economics Department. The instructor chosen. Dr. Benavie, 
is a highly personable, effective educator-“Committed to his 
students and his discipline. While he was open to experiment, 
he did not want the model, or indeed any system to come between 
him and his students. In the development of objectives his 
prime concern was to infuse his students with an enthusiasm 
for basic economic theory. He wished to interest and involve 
them to the extent that they would go on to elect further 
study in economics. 

On the surface. Dr. Benavie was an unlikely candidate 
for system development. His excellence did not seem to need 
improvement. Application of the model, however, revealed that 
while some of his goals were attainable, others were not being 
realized. When he and the team considered his planning and 
operation in relation to the model, the analysis showed 
specific areas which might be improved. Entry testing showed 
that his students came to him wirh a wide range of abilities. 

A significant number of students were without a workable 
knowledge of simple equations, the language of graphs, and 
ability to conceptualize tabular data. Post tests on the first 
semester showed that these deficiencies were not remedied 
during the course. The team suggested several strategies to 
resolve the difficulty. Programmed learning sequences were 
introduced on a voluntary basis for each ^f the areas. New 
methods of evaluation and feedback were developed to supple- 
ment the instructor’s subjective evaluation and enhance his 
rapport with students. 

In the inter-action with the team. Dr. Benavie developed 
a new awareness of; 

— methods of evaluation, 

— kinds of teaching strategies different from 
the formal lecture, and 

--“•the potential for the use of specialists. 

Because of his new awareness, he was able to comfortably 
modify his teaching to facilitate more effective learning for 
his students. His goals, too, changed to include new and 
specific behaviors that he could measure and that were conso- 
nant with his primary goals. Eventually, he made effective 
use of the programmed learning sequences developed by the team- 
one to provide a competency in simple equations, and another 
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to instruct his students in a working understanding of graphs 
and tabular forms. 



During the beginning period the team attempted to introduce 
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for the media to have a negative effect in that it came between 
the instructor and his students, and only v;hen he had carefully 
experimented and become comfortable at his owu pace with the 
elements of media, did they become effective. The overhead for 
a time was not a panacea, but an obstacle. Eventually, it 
became an effective aid. 



Throughout this process the team was not unaffected in the 
inter-action with the instructor; they began to realize a need 
for a more general sophistication and versatility. The media 
specialist became involved with evaluation; the learning spec- 
ialist with both media and evaluation; and the evaluator with 
presentation modes . 

The model itself was changed to fit the general considera- 
tions of the team and the instructor. It was obvious that in 
application such a model must be flexible. 

In sharp contrast to the operation with economics, the 
history professor choSen by the team was less than secure, a 
neophyte instructor, although an excellent history scholar. 

For Bill Bonds, the model provided an excellent developmental 
guideline. In this case, the model and the specialists worked 
to help shape his professional development. The security of a 
structure--that is, the model — and the assistance of the spec- 
ialists were a very real support to his teaching. In this case 
the specialists themselves became concerned for the retention of 
the decision function by the instructor, and it was the specialists 
themselves who strove to retain and value the human elements of 
teaching. Together the team and the instructor moved from the 
content statement as an objective to behavioral statements and 
beyond to develop the attitudes which comprise a sense of history. 
He moved out of the development of data — the chronological facts 
of history — into a conceptual development of the larger general- 
izations of history. Through the visual treatment as an adjunct 
to lecture material, we provided him with the opportunity to 
stand back and look at history with fresh insight. With a 
trainer -tester feedback system, he X7as made aware of what stu- 
dents were thinking and the level of student involvement at 
any particular time. He, therefore, achieved a more personal 
inter-action with students. What we did was to give new dimen- 
sions to the data of history, and to the instructor, an oppor- 
tunity to develop common generalizations. 



Evaluation became more than regurgitation of data. 

Dr. Bonds' whole testing procedure was changed through the 
use of the model. He moved from a mid-term and final test to 
a constant feedback and almost daily evaluation of concept 
development and a final analysis of the large ideas. 



With Dr. Bonds, we provided the directions for develop- 
ment and the structure v/ith which to organize his work. In the 
final analysis of our work with Dr. Benavie, we found that we 
had provided him with the impetus and encouragement to move 
out of the medi€ /al lecture mode so common to the university 
into new, dynamic, individual ways of teaching. 



Effects of the Project 



The project has had a significant effect on curriculum 
development at San Francisco State College. The procedures 
utilized in the Systems Development Project have been emulated 
in the School of Education for several classes, the Department 
of Nursing, and the School of Sciences. Specific curriculum 
revisions are occurring as a direct result of the project 
operations in the School of Social Sciences. Both History 
and Economics have actively moved toward course revision and 
vitalization. 

In both cases, the model was an effective device. In both 
cases, media was used to make teaching and learning more effec- 
tive. In both cases, the human values were not made subservient 
to hardware and system. 

We consider that the Instructional Systems Development 
Project has been a positive influence in the college and that 
its techniques v;ill continue to be used. The campus has become 
aware of the potentials for instructional improvement. We 
have demonstrated the potential of the Audio Visual services 
on campus. We have gained new insight into instructional 
organization. Most significantly, we now have at San Fran^^isco 
State College a team v/hich started out with separate specialties 
and has developed into a composite group which is highly compe- 
tent in the areas of human engineering, and human engineering 
is the ultimate requisite for the improvement of instruction. 
Finally, we find that the term "Systems" can be applied mean- 
ingfully and effectively to the very human acts of teaching. 



INSTRUCTIONAL DEVELOPMENT ACTIVITIES REPORT 



Michigan State University 



Background of Course Development Problems 

1. Prior to entry of Instructional Systems Development 
Project 

With rapidly increasing enrollments in Geography 204, 
the Geography Department has been faced with the decision of 
either maintaining the size of lecture sections at their pre- 
sent level or increasing the enrollment of the lecture section. 
Since there were practical and economical problems related to 
employing more faculty to instruct extra sections, it was 
decided to increase the size of the lecture sections provided 
that the following conditions could be met: 

instructional materials could be made 

available 

— lecture and recitation sections could be 
organized more effeciently 

a greater degree of coordination among 

the graduate teaching assistants 

The effective teaching of geography requires the pre- 
sentation of certain basic concepts, most of which fall within 
the categories of spatial relationships and man-land rela- 
tionships. To illustrate these concepts involves the taking 
of examples from any part of the world. Since this intro- 
ductory geography course is taught concurrently by at least 
four different instructors - each of whom uses a distinctively 
different approach to the course - it was difficult to organize 
the course along with its. instructional materials on a unified 
basis so that the same evaluation standards could be used on 
all the lecture sections during a given term. In other words, 
each geography instructor specialized in certain regions of the 
world, and therefore tended to draw his approach to the course 
from these regions. 

In general, the background of course development problems 
may resolve around the fact that no deliberate strategy has 
been developed in the past to identify coiranon elements for the 
course. Without these elements, it is difficult for these 






52 



instructors to identify concepts to be taught by all. 

This lack of concensus probably effects the identification 
of objectives which could guide the instructors in organi- 
zing Geography 204 on a mutually acceptable level. 

2. Entry of MSU*s Instructional Development Services 
and the Instructional Systems Development Project 

Michigan State University promotes a favorable 
climate for the purposable pursuit of educational change. 
Through the Provost Office the Instructional Development 
Service has been set up to guide and advise different 
departments in their efforts to improve courses of instruc- 
tion. Because ISD’s operational structure (i.e., EDP), 
many departments have had the opportunity to secure funds 
and consultant services in order that an innovative program 
could be tried out and then evaluated. 

Prior to ISDP involvement with Geography 204, the 
Geography Department and ISD had been involved in communi- 
cations relating to the improvement of geography. Early 
in October, (1965), Dr. John Dietrich, Director of Develop- 
ment Services at MSU, met with Dr. Sommers, Chairman of the 
Geography Department, to discuss the possibility of faculty 
members of Geography 204 working with the MSU’s ISDP team . 
Goals for Geography 204 were discusoOd as well as ways in 
which EDP and ISDP could be of assistance. 

At the present time, ISDP has produced, as well as 
secured, instructional materials identified as instructional 
needs by members of the geography faculty. Furthermore, 

ISDP has consulted and advised representatives of Geography 
204 as to an instructional strategy which might lead to a 
future extension of funds by EDP for course development . 

Plan of Action Applied to Problem 

1. Rationale 

Taking the view that people cuarged with educa- 
tional planning will follcn^ an orderly approach, i.e., sys- 
tem approach, it was thought that group planning behavior 
might parallel steps suggested in Ryan's System Approach 
to Educational Planning. That is, there might be an iden- 
tified trend which follows the phenomena of a system's 
approach. For example, before any decisions were reached - 
such as a central pool for slides, overlays and maps - 
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geography instructors were to prepare a list of "common 
elements" requiring media; this would be compatible to 
the first four steps of the Flow Chart of Trial-Procedures 
for Analysis and Design of Instructional Systems Employing 
Instructional Media. 



During March, 1966, a tentative strategy decision was 
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transparencies, and folding maps wM’:h would form a common 
basic collection for use by the Geography 204 instructors - 
would be secured and/or produred while members from geography 
and the project would seek ways in which to derive a listing 
of concepts. From these concepts, instructional objectives 
would be selected and defined. Then the developmental team 
and the geography representatives could develop an instruc- 
tional strategy which would identify the general conditions 
of learning expected fo"^ the type of learning represented by 
the objective. 



Because of the specialized interests of different geography 
instructors, it is asstuned that modules, i.e., standardized 
operational objectives employing specific media materials, 
explicating geography objectives would be developed. This 
would serve two purposes. First, flexibility would be 
provided so that each instructor might proceed not only at 
his own pace but via the area of his interest. That is, 
even though an instructor may approach Geography 204 from 
a different position, he can still use modules at the appropriate 
time when specific instructional objectives are required. 
Secondly, the use of modules would be packaged at a later 
date for laboratory use by students. 

2 . Procedures 

In terms of the Flow Chart, the following activi- 
ties have been identified. 

Steps One through Three - gathering of input: data 

Step Eleven - quick fix 

Step Three - gathering of input data 

Step Four - specifying of entry and terminal behaviors 
of course 

Step Six - the developing of a rationale for entry 
and terminal behaviors 

Step Seven - the planning of an overall strategy 

(e.g., the coordination of recitation 
sections by an assistant) 
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Step Three - the gathering of input data in regard 
to decisions upon future transmission 
vehicles 

Step Four - the specifying of entry and terminal 
behaviors 

In brief, the following acts have been completed: 

1. The "quick fix’* treatment has resulted in sets of 
geography slides for instructors as well as a complete 
master set located in the Geo£ -aphy 204 sector of the 
Geography Department . 

2. Several sets of overlay transparencies dealing 

with Europe and North America have been secured for the course. 

3. A variety of transparency maps as well as folding maps 
have been provided. 

4. During the 1966 fall semester. Dr. Matley was given 
release time to coordinate Geography 204. 

5. A part-time clerical vjork for Geography 204 was 
arranged and financed. 

6. Under the auspices of the Geography Department, 
assistants were charged with the equalizing of recitation 
work that is required for each enrollment section. 

7. An EDP proposal has been submitted to MSU's ISD 
in order to continue the development of Geography 204 
during 1967-68. 

8. A strategy has been suggested to Matley as to bring- 
ing about a consensus that pertains to geography concepts. 

9. Operations are underway to gather data as to students* 
expectations of Geography 204. 

In conclusion, the overall strategy has been toward the 
improvement of large class instruction, e.g., slides and a 
master set, indexes of media materials, and an identified 
sub-system for coordinating Geograp>hy 204. The second phase 
deals with the attention directed to the recitation classes 
and to the instructor’s opportunity to have access to modules 
of media resources built around identified concepts and objec- 
tives. 
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Competencies of Specialists Involved 



The development team is viewed as consisting of the 
following: 

---Geography Coordinator 
— Learning Evaluator 
— Media Specialist 
— Media Coordinator 

Each will be treated separately as to his role and 
function. 



1. Geography Coordinator 

His role is perceived by the others as having aca- 
demic competency in geography as well as direct channels of 
commun-.cations with the geography instructors. 

His function is to understand and diagnose the problems 
of his geography colleagues in order that he can interpret 
their problems and needs to the ISDP members. At the same time, 
he is charged with facilitating change with his fellow instruc** 
tors by setting up and implementing organizational operations. 

The Geography Coordinator is the link between the ISDP 
members and the instructors. The instructional strategy 
developed by the team is only as good as it is conveyed back 
to the department’s instructors. 

2. Learning Evaluator 

His role is perceived by the others as having pro- 
fessional judgment in regard to identifying for each objec- 
tive the type of learning required. This expectation, shared 
by others for the learning evaluator, does not exclude them- 
selves from entering into this domain. 

The function of the Learning Evaluator is to guide the 
collected descriptive information into prescriptive policies. 
That is, it is his function to help the Geography Coordinator 
identify what is to be done in order that operational plans can 
be set up so that at different stages of planning and develop- 
ment-decision phases can be formed. In other words, he enables 



thfe others to plan schedules and sequences of events so that 
previous elements of the instructional strategy, when testeo 
at decision phases, maximize performance of the projected 
objectives . 

3. Media Specialist 

His role is perceived by others as to identify the 
nature of the unique quality of the stimuli needed. At the 
same time, he is expected to have knowledge of the character- 
istics of each medium for presenting each stimuli identified 
by the Geography Coordinator and the Learning Evaluator. 

His function is to clarify - from what has been tenta- 
tively planned - the conditions that a specific stimuli is 
needed. For example, it might be that the specifications 
required for a particular material linked with a medium may 
not be sufficient for the recitation classes although it may 
be necessary for the large class instruction. Additional 
functions of the Media Specialist relate to his understanding 
of educational taxonomy so that he can actively cooperate with 
the Geography Coordinator and the Learning Evaluator in making 
decisions as to the special characteristics of the stimuli 
(e.g., verbal or nonverbal, intensity and duration of the 
message, movement and/or special technical effects). 

4. Media Coordinator 

His role is preceived by the others as transmitting 
production decisions to their particular destination. It is 
expected that he can give the others a highly reliable esti- 
mation as to when production materials - either secured or 
locally produced - will be available. (He and the Media 
Specialist are responsible for estimating cost factors.) 

His function is to serve as a liaison between the Ge- 
ography Department and the Instructional Media Center. 
(Providing and arranging media services to the Geography 
Coordinator is tantamount to feedback for the planning and 
development group.) An additional function of the Media 
Coordinator would be in the area of handling Procedural 
Evaluation Review Techniques in order that opportunities for 
articulating the sequence of instructional events are 
maximized. 



In summary of the competencies involved, it is thought 
that the Geography Coordinator represents the "What" in 
terms of goals; the Learning Evaluator represents the "Why" 
in terras of guiding the planning and developing into opera- 
tional behaviors; the Media Specialist represents the "How" 
in terms of under what conditions a specified stimuli should 
be linked with a particular medium; the Media Coordinator 
represents the "When" in terms of implementations of instruc- 
tional materials. 

Sequence of Events That Actually Resulted 

1. A quick synopsis 

The matching of sequences with the suggested flow chart 
indicates that after step three, i.e., the gathering of input 
data, step eleven, i.e., the securing and producing of speci- 
fied instructional materials, followed. 

It would then appear that from step eleven a recycling 
occurred. That is, the attention of the team was again con- 
centrated in steps three through seven. 

2. A closer view 

It might clarify the reader’s perception concerning 
the sequence of events if certain steps along with its con- 
ditions were considered. 

To begin with, step one was never actually considered. 
Rather, it was a given condition. In other words, university 
policy concerning the geography course vzas more interested in 
the fact that a course existed rather than how it was to be 
planned and developed. Thus, Che team faced itself with a 
traditional climate which indicated that tne course vzas unique 
unto each of its instructors. The point presented now - to 
be developed later on in this section - is that any directed 
energy would be received with subtle resistance. 

In relation to step tvx) , it would appear that the Geography 
Department received the talents of the ISDP team as a means 
to an end. That end is the further financial assistance of 
EDP in improving the 204 course. This may appear as a para- 
dox since the above paragraph suggests resistance by the 
staff. To be sure, the resistance referred to is unintentional# 
but nevertheless it is there. 



Step three had never been operationalized until Matley, 
during February, 1967, had undertaken the necessary leader- 
ship to gather a ranking of concepts shared by the course 
instructors. This undertaking is regarded as the sine que 
non for an instructional strategy. 

Meanwhile, it must be remembered the university courses 
do not wait for new ideas. Rather, these "Innovations” 
assimilate bit by bit into the ongoing instructional process. 
Hence, the so called "quick-fix" v/as an accepted part of an 
overall strategy. This last statement can be supported in 
terms of the general purpose of the Geography Department 
seeking aid: 

The purpose of utilizing the talents of EDP and ISDP is to 
improve the instruction of Geography 204. Note that the 
materials secured and produced for the quick-fix did just 
that, i.e., improved the instructional communications of 
each of the instructors. Therefore, it is sufficient to 
say that each instructor felt to a degree satisfied that 
his communications to his large class was enhanced by rr-- 
ceived instructional materials. 

The above paragraph is not to be confused with the 
development of course obiectives, i.e. instructional strategy. 
The development by the Geography Staff of instructional objec- 
tives has been and still is a slow process which is character- 
ized by the matching of sequences with the first eight steps 
in Ryan's Flow Chart. That is, in terms of the suggested 
fifteen steps, the trial procedures for analysis and design 
of instructional systems employing instructional media has 
not passed the eighth step* 

3. A suggested interpretation of the quick-fix 

It was suggested before that indirect resistance 
existed. The assumptions to this speculation are thv’: following: 

a. The course had been accepted as being taught 
concurrently by instructors each of whom had 
a unique approach to the introductory course. 

b. Although the Geography Department wanted to 
improve the course, each instructor faced a 
challenge to his autonomy by the fact that an 
analysis and design of Geography 204 for the 
utilization of instructional media forces a 
consensus as to concepts. 
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c. " The climate of change at MSU has influenced 

the Geography Department, but at the same 
time has conflicted with traditional instruc- 
tor expectations, (e.g., a specialized 
approach to conducting Geography 204.) 

d. The initiators of chatige for Geography 204 

are not perceived by the faculty as coming from 
within their department but rather from the 
outside, i,e., EDP and ISDP. 

With these assumptions, the significance of the "quick- 
fix*' is interpreted along the lines of Everett Roger's charac- 
teristics of an innovation. 

First of all, it is thought that the "quich-fix" repre- 
sents an innovation - new idea. It is the key for gaining an 
awareness of the faculty as to giving major consideration to 
improving the instructional objectives of the course. 

As Rogers states: "It matters little whether or not 

an innovation has a great degree of advantage over the idea 
it is replacing. What does matter is vzhether the individual 
perceives the relative advantage of the innovation." 

The point taken is that the "quick- fix'* innovation opened 
the opportunity for the sophisticated "instructional strategy" 
innovation, which is now in process. To support the statement 
that the quick- fix is an awareness factor which cracks the 
unintentional resistance apparently held by the instructors, 
five characteristics of the quick fix are briefly discussed: 

1, Relative Advantage 

Relative advantage is the degree to which an inno- 
vation is superior to ideas it supersedes. In this case, 
the quick fix. resolved the problem of increased enrollment 
by enabling visuals of various sorts to complement the 
lectures. 

At the same time, the quick fix represented found 
money, since these materials were not at the expense of the 
Geography Department. 
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2. Compatibility 

Compatibility is the degree to which an innovation 
is consistent with existing values and past experiences of 
the users. The use of provided instructional materials still 
enabled each instructor to use only those materials which 
served his instructional purposes. Hence, the use of the 
quick fix was suggested by the department and at the same 
time useful to the instructor. Hence, no conflicting values. 

3. Complexity 

Complexity is the degree to which an innovation is 
relatively difficult to understand and use. Instructors 
found it easy to locate materials at a designated section 
in the department. Each had their own set of slides. 

Equipment used to transmit the materials were already 
stationed in the lecture rooms. Thus, there vzas little 
complexity to the use of the quick fix. 

4. Divisibility 

Divisibility is the degree to which an innovation 
may be tried on a limited basis. Because of the range of 
instructional materials, instructors could try out samples 
of the quick fix. That is, they could set t?'.eir own pace 
in experimenting with their lectures. Rather than all or 
nothing, bits at a time could be tried out. 

5. Gbmmunicability 

Communicability is the degree to which the results of 
an innovation may be diffused to others. The quick fix was 
easy to describe to others. Each instructor could relate his 
experiences to the other. At the same time, recitation 
sections could indirectly confirm whether or not the use of 
selected instructional materials enhanced the responses of 
the students. 

The view of the quick fix is thought to be that of 
setting the stage for the receptiveness of the Geography 
faculty in regard to their analysis and design of an 
instructional system employing instructional media. 
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Outcome of Development Activities « Resolved and Unresolved 
Issues 



1. Resolved 

Tantamount to the development activities carried on 
so far is the realization by the Goegraphy 204 instructors 
that there must be 'initial goal priorities set up. These 
priorities are the instructional concepts for the course, 
e.g., ‘'common elements." Even though there was a need for 
the quick fix, it became apparent that a base was needed for 
developing an instructional strategy. Speculating on the 
last statement, it seems that the quick fix forced the 
attention to the cliche: "Now that we got it, how can we 
use it?" In other words, an awareness climate was provided. 

As to the activities set in operation so far, logs ten, 
eleven, and twelve can provide information. More important 
is the fact that ISDP involvement enabled "housecleaning" to 
occur in operational policies for Geography 204. For example, 
coordination action is being carried out in regard to the 
instructional policies of the recitation classes, e.g., amount 
of readings, same number of tests; interest in being directed 
towards the incoming expectations and the outgoing attitude 
changes of students. 

Although ISDP activities terminate "rhis spring, it appears 
that Geography 204 has begun developing its own inertia. The 
planning and development activities will continue provided 
that leadership is implemented and financial support, e.g., 

EDP, is forthcoming. 

2 . Unresolved 

The talents of the developmental team are only as 
effective as the people who contemplate their suggestions. 

A systems approach is sufficient for constructing and develop- 
ing ail instructional strategy, but a willingness by the 
Geography members to form a temporary social system is a 
necessity for substantial improvement to occur. For tempo- 
rary systems, one should consult Chapter 19 of Miles Innova - 
tion in Education . 

The point is, there are two typei> of systems involved in 
an ISDP. There is the systems approach which deals with the 
following: 
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- — defining performance goals 

— translating these goals into sub-systems 
of general and specific functions 

— specifying the means of executing these 
functions and defining the components of 
the systems including human capabilities, 
machines, and materials 

— -distinguishing between functions that can 
best be performed by persons with known 
competencies and functions that can best 
be performed by instructional materials 

— planning schedules and sequences of events 
so that all components of the designed system, 
when tested and retested, maximize perform- 
ance of the system as a whole in accomplishing 
projected objectives. 

It is believed that this, i.e., the above, is sufficient 
condition for change to occur in the discipline. However, the 
temporary system encompasses the interactions identified by the 
systems approach. The human involvement of receiving and then 
acting on the planning and development activities cannot be 
left to "good faith." That is, there is unresolved issues 
to the setting up and the implementation of the faculty *s tempo- 
rary system. For example, there is no reliable data to answer: 
"What are the restrictions imposed by reality which cause the 
instructors to take a passive role in working with the 
Geography Coordinator?" 

Hence, the unresolved issue centers on whether or not the 
temporary system is a necessary condition for a systems approach - 
which leads to the development of an instructional strategy. 

To rephrase this concern, one might give attention to 
those questions: 

Why must a year elapse before the Geography instructors 
identify common concepts for their 204 course? 

Why does the Geography Coordinator find a delay in 
implementing a policy with his fellow instructors? 
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Are the decisions made by the team, i.c., Geography 
Coordinator, Media Specialist, Learning Specialist, the 
same decisions that are carried out by «the instructors? 

Suggested directions in regard to unresolved issues: 

1. Establish necessary and sufficient conditions for 
systems development. 

Example one -- It is necessary that there exist a 
temporary social system - which can be distinguished 
by its input, process and output characteristics - 
in order that decisions are acted upon. 

Example two — It is sufficient that a systems approach 
can be carried on by a "team*' which will devise an instruc- 
tional strategy. 

2. Establish measurements which can trace the flow of 
communications from the team to the instructors and from the 
instructors to the team. Perhaps in this way insight can be 
revealed as to the how and why that decision.^ are negated, 
modified, or accepted. 

The concluding statements regarding the analysis and design 
flox 7 paradigm centers around the organization. This speculation 
might be given impetus if one lists under the organization sub- 
groups within subgroups, i.e., concentric rings. When a number 
of subgroups are identified as those coming under the auspices 
of ISDP, then one should attempt to investigate what subgroups 
are left out but contribute controls necessary for gathering 
input data; steps two and three of the model. In other words, 
it is suspected that more information is needed as to steps 
one through three than are actually collected. Perhaps 
systems work could be made to produce more efficient results 
within a shorter range of time if such information were 
collected. 



A. Comparison of kind and sequence of development steps 
employed with those states in hypothetical model. 

From previous reports, it was indicated that a ''quick 
fix" was undertaken as a temporary measure. The identification 



64 



and selection of those instructional materials were pre- 
sumed to be undertaken by the faculty members involved 
with Geography 204. Supposedly, those officially con- 
cerned with the course were to proceed with planning 
and development. 

At the start of the fall semester, there was no 
\ 7 ritten evidence of faculty improvement in regard to curric- 
ular decisions. During the past months of consultation 
with Matley, it appeared that his efforts to identify con- 
cepts had resulted in "awaring** the faculty of latent pro- 
blems that are associated with surveying the course. 

Although there has been limited success in securing 
specific areas of agreement in regard to the **whats" and 
“hows" of instructing Geography 204, there has been 
administrative success in setting up policies regarding 
the quantity of student work. 

It is suggested that while the developmental team 
was involved with guidelines for planning an instructional 
strategy, the faculty members were involved with basic 
housecleaning details. In terms of the flow chart, it 
might be construed that the developmental team was interes- 
ted with curricular objectives while the Geography 204 
faculty was interested (passively) with administrative 
objectives. 

Perhaps a decision stage should have been built into 
the flow chart between the gathering of input data and the 
specifying of behaviors. 



B. Estimate of the general iz ability of developmental 
model to other higher institutions pursuing in- 
structional development. 

1. ISDP model as is: 

a. The model does not distinguish the students 
as being either in the system's environment or 
of the system itself. 

During the past meetings, discussions were directed 
towards identifying the students' expectations from the course. 
Not only was there interest concerning what the student expec- 
ted to get from the course - in terms of cognitive information - 
but there was academic concern as to the students' affective 
change . 
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Although Matley ^^as impressed with these concerns, it 
appears that since his faculty has had no data concerning 
their audience, the students’ expectations were not taken 
to be an input for the analysis and design operations. 

b. The model is a logical flow pattern that one 
would expect to be carried out under ideal condi- 
tions. It is a linear description that leads to 
further probes of a protein society. If one 
accepts the concept that a system permits the 
meaningful identification(s) of sub-systems, 
then one may be optimistic about the values of 
the flow chart since it heuristically leads to 
the revealing of a heirarchy of sub-systems 
and their various social structures within. 

In short, the flow chart enables a team to 
begin working where the clients are. 

2. ISDP model with some modifications: 
a. Specify changes 

It is suggested that when a courses* concepts 
are not consistent, and/or mutually reinforcing, 
there are sub -systems that were not considered. 

If these components are not identified then the 
flow chart lacks integration. For example, 

Matley *s undertaking to produce Geography 204 
concepts for faculty consideration was met with 
barriers of communication; this barrier was 
based not on content as such, but on how the 
content shold be presented. 

If the purpose of the Instructional Systems 
Project is to in^rove the flow chart, then one 
might consider the refinement of the "gathering 
of input data stage" and the "dry run-through- 
field testing stage." 

Under "gathering stage," there might be a need 
to specify consensus agreement concerning so much 
of a course. If there is lack of agreement then 
there might be several configurations to aid in 
bringing in additional information (e.g., depart- 
ment policies; national curricular policies; 
student expectations.) 
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Specifying terminal behaviors can only begin 
when the input stage has been given a simulated 
dry-run through. Concurrently, some instructional 
aids can be secured which do not affect the ongoing 
curricular decisions. Such aids would be those 
that improve the existing lecture presentations 
(e.g., fidelity of communications). 



PROJECT EVALUATION TEAM REPORTS 



The general design of this study called for periodic 
reviews of the instructional development activities by a 
team of four independent observers. The Project Evaluation 
Team. 



These evaluators obtained access to the work in the 
development proceedings at the four demonstration in- 
stitutions by means of tape recordings of discussions, 
transcriptions of tapes, activity logs, and the four 
project review meetings with team members, as described 
in Chapter II. 

On the basis of these data, the evaluators have 
prepared four reports encompassing certain objectives of 
the study. They are presented here as follows: 

1. Analysis of the Data Secured on Actual Development 
Procedures - Dr. Phil C. Lange 

2. Heuristics of Media Implementation in Instructional 
Systems Development - Dr. John B. Haney 

3. Usefulness of the Development Model for Generating 
Research Hypotheses and Specifications for Specialist 
Training - Dr. John M. Gordon 

4. Dissemination Value of the Demonstration Approach - 
Dr. John Childs 
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ANALYSIS OF THE DATA SECURED ON ACTUAL DEVELOPMENT PROCEDURES 
AT THE FOUR COOPERATING UNIVERSITIES 

Dr. Phil C. Lange 
Data-Gathering Strategies 

As previously mentioned, the decision-making discussions of the 
Instructional Systems Development teams at the respective universities 
were recorded on audiotapes. The circumstances on some campuses were 
such that members of the team had chance meetings and discussions at 
other times than their schedule^ tape-recorded meetings. And some- 
times progress was made along decision points at these chance inter- 
changes that were not recorded. Nevertheless,, the tape recordings 
were highly revealing in showing the range of different ways that 
the different teams gradually moved through the steps of the model, 
although seldom was this movement by a smooth, direct path as 
sequenced in the ISD model. The teams often moved around frustra- 
ting decision points, jumping ahead to more familiar ground, and 
then backtracking later. For example, in the first year of the 
project, this was typically the case when teams tried to begin 
with a fast, direct approach to the initial analysis of course 
objectives to be stated as student behavioral outcomes. In such 
instances, as many as three of four consecutive sessions made no 
progress in detailing behavioral outcomes. Yet essentially the 
same team members working later with other instructors avoided 
this "hang up" in discussions about specification of behavioral 
outcomes by going directly to the end-of-course test items or 
by running through the steps of the model at a low level of 
approximation; and then recycling through the steps again to 
establish better specification and precision of decisions with 
each cycle. In short, as the teams became experienced, the 
tapes indicate that the media specialist and the coordinator 
and the evaluation specialist became much more capable in getting 
more data from the professor at the in -put stage and then making 
from this a tentative translation to behavioral outcomes and 
operational procedures. 



In the early tapes there are occasions where the professor 
could not or would not (or even fought against being asked to) 
describe student behaviors or performances. But he was eager and 
willing to describe and analyze his teaching behaviors ; and the 
coordinator or media specialist seized on this behavior as a focus 
for instructional improvement. When this occurred, invariably 
the discussion tended to move rapidly and verbally through the 
model (if one interpreted the steps as describing the teacher's 
past behavior). Such "input" data usually established a bridge- 
head: the teacher's satisfaction or dissatisfaction with student 

results or attitudes. On such a bridgehead the evaluator or 
coordinator might bridge over to what each student would need to 
do in the course the team was improving. 
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Whereas, a tape recording of any single session of an ISD team 
often presents the listener with a confusing problem in placing 
precisely the discussion threads according to the ISD model, the 
full set of tapes of a team's session reveals the high compatibility 
of the model and the actual, decision points recorded. As the 
p3irti.ci.p3nts till 8ms gLvbs 1i 3.V8 S8i.d> i.n iT8tircsp8cti, thG ISD modsl 
serves as a map. Thus, some of the early sessions (at the "input” 
point on the ISD model) seem to have discussion topics that verbally 
touch on practices and problems that appear all over the model. 

But while these sessions sometimes make no forward progress to the 
next decision point on the model, the ISD model serves like a map 
to encourage more deliberate attention to "input” so as to be 
better prepared for what probably lies ahead. 

The tapes objectify many of the team conclusions, for example -- 
that the "input" step is a most important step. Getting the necessary 
facts here pays off in smooth sailing later. The tapes on "input” 
discussion bring alive the simple idea that excellent college teachers 
may actually know very little about the students they will have to 
teach, or the facilities they will have, or the curricular validity of 
end-of-course evaluation. When an instructor honestljj immediately 
admits, "I don't know the, background of students," "All I want to 
develop in the student is a life-long excitement for this field," 

"We devise a final examination that spreads student scores so that 
their marks range something like a normal curve," "I think our 
course would be better if we used more visuals," — the next step 
to "specify entry and terminal behavior" on the model seems far away. 

Another revelation on the tape recordings was the different 
styles of decision-making in which individual team members functioned 
in their team relationships. Interesting!}^ the variation of a 
different professor in a team might greatly affect the style of 
operation of the other team members; and the style or pattern in 
which one of the specialists functioned in the initial session 
tended to persist through several sessions. For example, in one 
team, the media specialist was consistently the member who began 
early to press for specification of behavioral outcomes and opera- 
tional descriptions whenever the professor kept speaking to the 
matter of audiovisual aids and equipment. Yet* this same media 
specialist functioning in the same semester in the opening sessions 
of another team (with a professor who felt he was maneuvered by 
his administration into having to do something about his course) 
functioned as the member who baited the instructor along by a 
chained revelation of the assistance and "goodies" that might 
accrue to the professor as he took steps to improve the instruction. 
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Most evident in the tapes themselves are the frequent, lengthy 
interchanges that do not result in decisions. As one professor 
remarked, "a team may have to spend a lot of time getting to know 
one another before an instructor is ready to reveal what results 
he wants in students and how little evidence he operates on." 

Also, something happens to individual judgments between taped 
sessions. Quite often a session begins v;ith a happy resolution 
of a problem or issue that was the troublesome, unresolved and 
sometimes emotional ending of the previous session. In fact, the 
team seemed to become skillful and intuitive in closing a session 
by calling attention to their unfinished agenda items (as if 
planting the bulb that blossoms at the next sessioit) . 

Further analysis of the tapes for their “decision^moves" : 

The tapes have not been analyzed for the gamesmanship of inducing 
and avoiding decisions — except as some broad guidelines are 
stated in the "heuristics" for practitioners (see page 79) . Such 
an analysis, however, might be applied to these tapes, for it is 
quite apparent that when a different professor is a member of a team, 
the other members quickly draw upon different resources in their 
repetoire of group strategies and meld their combined involvements 
into a new strategy for moving toward decisions. This applies to 
the team when it is mature in its sixteenth session as well as in 
its earlier sessions. The "sparring" that some participants make in 
early sessions is almost entirely "counterpunching" or "responsive," 
whereas other participants spar mainly with initiatory thrusts, plus 
more thrusts, and more probes. 

ISP model helps make sense of tapes ; For the purpose of this 
project, the tapes do show in the long run — allowing for information- 
al scoop-backs and timely advance scouting — the decision-making 
procedures of the team roughly approximate the steps and the sequence 
in the m ^' del . However, direct discussion of the model itself, or even 
its steps in the early sessions, tend to confuse and delay the actual 
decisions. But, it was not the purpose of the project to discuss the 
model to ascertain its fitness. Thus, it is reasonable on the basis of 
the taped records to say that the model had a "general" or a "condi- 
tional" fit in the university settings. This is especially true when 
the team uses the model as a map to enlighten its activities and 
progress . 

(a) Tape abstracts : The tapes were also subjected to analysis 

whereby they were abstracted for the questions posed, the decisions 
reached, the sequence and relation of questions and decisions, and the 
team member initiating the action. As already noted with reference to 
taped records, these abstracts also showed the same patterns while 
losing the emotional overtones and stress that are so apparent in the 
tapes themselves. The abstracts, however, make it very easy to see 
how each team member may carve out a functional role or be rifling 
in on a decision he wants. The abstracts, for example, show that 
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when the evaluation expert joins the team for the first time there 
tends to be a radical change in the content of the questions, in 
the shared leadership in the group, and the participation of the 
group members. In several of the team records, the evaluation 
specialist joins the team either during an impasse with, or a 
detour around, what the model labels "specifying entry and terminal 
behaviors." His presence is the occasion for getting at the same 
thing from the bottom end: "Are you satisfied with the evidence 

of what your students can do at the end of your course?" 

(b) Expectation and reaction forms ; Another written record 
available in the first year of the project was an expectation- 
progress and reaction form filled out by participants before and 
after each team session. Later this procedure was replaced by 

a log kept by the coordinator; the shaping of the individual 
expectations was replaced by shared planning for each session 
among the coordinator, media specialist, and evaluation specialist. 
The obvious thing about data from the expectation and reaction 
forms was the degree to which everyone overestimated the expected 
progress from sessions getting at "entry and terminal behavioi;" 
an extent to which these early sessions therefore were rated as 
accomplishing "much less than expected." Although the use of the 
individual expectation-reaction forms were generally discontinued, 
there is enough basis tc indicate that the professors pretty 
generally underestimate the time, care, and information that must 
be invested in the input stages and agreements on outcomes; and 
some other members too are consistently overop tomis tic even though 
they are more likely to be satisfied with "less than expected." 

As the teams got older, all members became more realistic in expec- 
tations and satisfactions. 

In some further study that might wish to analyze more rig- 
orously the gamesmanship of decision making, the use of something 
like the expectation and reaction form would seem to be essential; 
for there are several instances where a team member who was 
basically a "counterbuncher" or "responder" was almost a non-parti- 
cipant in the entire discussion session when his "expectations" 
had not been introduced or thrust forward by other members. 

(c) Summary logs : In the second year each team coordinator 

wrotLB summary logs following each team session. The logs frequently 
included editorial-type observations and judgments, and often the 
logs identified or even elaborated important factors of school 
politics. Or the log might include the coordinator's guesses 
about hidden agenda, or venture an interpretation of why a team 
member did what he did. These are the kind of data that are not 
included in the abstracted report of a taped record. Thus the 
summary logs often make explicit the importance of political factors 
that a listener could only imply from an edition of the tapes. In 
short, such factors as peer pressure, administrative support. 
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prestige relationships, etc., that reinforce instructors when 
they make changes to improve instruction are more quickly noted 
in the summary logs than in the tapes or the tape abstracts. 



(d) Quarterly and terminal reports : The quarterly and 

terminal reports of the teams and the occasional national conf“ 
erences among all teams and project personnel arch over longer 
time periods and evidence a mellowness and a gross accumulation 
of much change and highlighting, which is not so easily apparent 
in reports of individual sessions. Thus, long-range reports 
rather consistently say that the ISD model fits their develop- 
mental process, at least conditionally. Typically, the professor 
reports with hindsight that it was tough going (or he wanted to 
quit) when the pressure was on clarification of entry and terminal 
behaviors -- but in retrospect it was the best kind of discipline 
for him. Other professors are equally adamant in saying that 
someone other than themselves can translate their teaching behavior 
into behavioral specifications for students. The more experience 
the teams had with the systematic improvement of instruction, the 
more generally did they apply the XSD model to their efforts and 
the more helpful did they consider it to be. 



These long-range reports are the excellent source for many of 
the "heuristics" which might be researched as high probability 
hypotheses. In the meantime, these "heuristics from the combined 
experience of the four ISD teams can serve as guidelines for those 
who wish to apply an instructional systems development approach to 
their own projects for instructional improvement. 



Procedures and the Hypothetical ISD Model 

The primary function of each team effort was to improve 
instruction. A correlative but secondary function was to test 
the applicability and usefulness of the ISD model. There are many 
ways that persons can learn from or use a model, to list somet 

(a) by adopting the model as is (rather unquest ioningly) 
to be "plugged in" like a unified module for operation 
as a sub-system in the overall operation. 

(b) by exploratory applications and adaptations of the 
model, along the approach of "action research" 
(scientific "if-then" problem solving). 

(c) by creating counter models , building better models 
(comparison) . 

(d) by creating a generalized model from their own 
experience (inductive arrival at an "experience 
model") « 
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(e) by critiquing a sequence or matrix of decision-making 
actions with REFERENCE TO THE MODEL. 

(f) by generalizing about UTILIZATION of model introduction. 

(g) by reading ) 

hearing ) how the model was used 

viewing via media) 

(h) by seeing ALIVE how another person uses the model 
(one dissemination aspect of this project). 

Actually, the data in this project could be clustered about these 
natural polarities with reference to modeling. In this listing, 
S.» ^ were part of the evaluative and dissemination 

aspects of this project. 



Adopt or adapt or generate anew is a crucial issue in a 
through d. This aspect is discussed further by R. Lippitt Tn 
American Journal of Arthopsychiatrv . July, 1965, 683 pp, Lange 
in NSSE, 1967, Yearbook: II. 



The data in this project would indicate that at the beginning 
the participating teams tended to perceive the model as fixed 
and as a rather rigid prescription; this rigid prescription was 
not adopted. The same teams later were viewing the same flow 
chart as generalizing the steps and the sequence as being flexible 
and viable with local conditions; and this viable perception was 
accepted as tenable. Thus, the model may be changed subtly by 
level of understanding as well as by a formal change in kind. 
Evidence of this is the fact that the teams consistently found 
the model more acceptable, while at the same time they more verbally 
criticized its lack of detail and the prospect of pluralistic 
refinements and loops, even while simplifying the model to fit 
formats and audiences . 

Readiness is the over-riding factor, while the underlying 
problem is how to diagnose for readiness and initiate level of 
treatment. Readiness and diagnosis for readiness apply whether 
the reference is to the teams’ understanding of the model, or 
the teams' application of instructional systems development to 
a college course. In short, there is some evidence here to indi- 
cat'^' that (a) initially modal adopt io n is more difficult than 
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model adaptation , (b) later when a group matures (has worked 
together long) it is more in an adoptive mood, but adoption 
may imply interpretations. From his experience, S. M. Corey 
in. Helping Other People Change , notes that when groups begin 
working together, their initial energies are mainly directed 
to interaction with each other and interpolations: later with 
maturity the group is more actionable and more receptive. 

As to diagnostic procedures, it is true that the project partici- 
pants have verbalized their hunches, rules-of- thumb and sensi- 
tivity indicators; but the evidence shows that these must be 
subjected to the action research of empirical efforts. 

When not to use ISP; when not to apply the model : This project 

set out to apply ISD and to test the usability of the ISD model. 

It did not set out to test its no limits. The energies were 
committed to making it work, but the data are filled with evidence 
of withdrawal feelings, concern about the material and human 
costs, and the importance of selection. Thus, it is extremely 
important to stress that the ISD model is a change model — a 
production model ; theoretically, it is inappropriate where there 
is no product or where the idea of the product is not communicable. 

The language of the model also implies change, not only in 
the instructional moves, strategies, and materials, but also 
changes in the instructor and the students. This introduces an 
ethical question when the instructor, for example, does not want 
to change and enters the ISD plan with no awareness that the 
system is set to change him in some way. His right to be fore- 
warned of the risk of change and his option not to begin need to 
be respected. 

The ISD model is a man-and-materials system : It is not to 

be confused with systems development in those technologies where 
reliability is paramount to the exclusion of human variability. 
Moreover, it is a system designed for higher education where 
there is a tradition of academic freedom for instruction and 
an expectation that students can be articulate and mature both 
about their learning and their processes. 

The ISD model lacks automatic ejection outlets for wastage ; 

This ISD model differs from many production models (as in a 
factory production) in that it shows only what continues on 
through the system. It shows neither what gets trimmed off as 



waste (discarded objectives, assignments, etc.) or what may have 
been converted along the line into by-products (research skills 
in graduate students, attitudes in team members, etc,). And to 
this extent, it makes cost accounting more difficult. But ejection 
when it occurs in this model must hinge on human judgment at the 
time, tempered by experience. From such experience guidelines may 
develop. Some objectives, some demands, some instructors could 
ruin this system or this system could ruin them. The implication 
for the application of the flow chart is that its users must apply 
human judgment along the way at check-points to screen out soue- 
thing that shouldn't be going through the system. The othei side 
of this quality control along the way acts to keep the materials 
and humans within the interaction of the system when there is 
high probability of gain. 



The data also evidence difficulty in putting value on change. 
This is distinct from determining cost. For example, one team 
might be working to make technical and quali^'.y- control refinements 
on already excellent instruction, while another team might be 
working to bring direction to an otherwise aimless ineffectual 
course. Which change is more worthwhile ? Which changes should 
have the highest priority ? With which type of change at what level 
of performance should a university team begin, and when should it 
give up in the absence of what results ? 

Implications for application of the model ; It is interesting 
that the participating teams chose n£t to spend energies on elaborate 
or precise revisions of the model. As noted in their terminal 
reports, they found a working interpretation of the ISD model for 
their situation. Thus, there seems to be some basis for concluding 
that eventually the general nature of the model contributed to its 
success. Nevertheless, in their initial use, some users had comments 
or criticisms like these about the ISD model; 

(a) The ISD model implies more sophistication in evaluative 
rationale and course objectives than is actually the case. 

(b) The ISD model gives little help in deciding when and 
where to start. "Begin" contributes little to the flow chart. 

(c) The campus politics and the administrative organization 
for change probably have more importance at the "input" step than the 
model recognizes . 

(d) The great difficulty in dealing directly with "entry and 
terminal behaviors" is not made adequately evident in the flow chart. 

(e) The ISD m.odel actually implies that the learner and his 
learning gain must be a consideration all along the steps of ISD; 
but this is not as evident in the rationale of the model as it 
might be. 
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(f) The rationale for the membership of the ISD team 
perceives an "improvement of instruction role" for the media 
specialist, whereas the local image may be that of an audio- 
visual service that purchases equipment and provides requested 
materials. 

(g) Some subject matter areas, some levels of instruction 
in areas, and some methods would seem to be easier applicators 
than others; but the model seems to make no differentiation among 
levels or strategies. (Later the same teams no longer raised 
this kind of criticism.) 

(h) The differences in instructional styles and in the 
learning styles of students are not adequately recognized in 
the ISD model. (Again this was an early criticism which team 
members no longer made as they became more experienced with the 
model.) 

(i) The model is too detailed and its language a jargon 
that need not be imposed on the professor and his colleagues. 

Other Wavs of Looking at Instructional Systems Development 

The ISD model directs attention to the operational relation- 
ships of decisions being made to effect instructional changes. 

These same decisions could be reviewed to see how they might fall 
into three domains: administrative decisions (e.g., who shall be 

educated, how long, at what cost, in what facility, with what 
personnel, etc.), curricular decisions (e.g., what shall be taught, 
when, to what level of proficiency, etc.), and instructional 
decisions (e.g., how to manage the environmental factors so as to 
bring about the desired learnings). 

This analysis might then be correlated with the theoretical 
roles of evaluator, coordinator, media specialist and subject- 
matter specialist, — and with the actual pre.ctice in each team. 

The frequent discussions about "campus power" and "campus 
politics" would suggest that a sociological or political science 
study of the power factors in the decision-making processes in 
ISD would reveal decision points outside the perimeter of the 
evident ISD steps. Similarly an economist would very likely 
reveal that for economic reasons many degrees of freedom are 
lost or not even considered as alternatives at decision-making 
points in the ISD model. As noted previously, he would very likely 
inject into the input some tentative formulas for relating worth 
of change and the cost of change to specific cut-off and feed- 
back points along a linear model like ISD. In their own differ- 
ent ways, the anthropologist and the social psychologist can see 
other reasonable relationships in instructional development. 
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Yet it is noteworthy that once having been exposed to the 
ISD model during two years of their own instructional system 
development, these participants found the ISD model workable 
for its fundamental steps and its basic structuring of those 
steps. The ISD model, however, took on coloration from their 
experience; and the ISD model as they finally recousnend it has 
subtly changed its perception as it became alive for them 
with the following ’’heuristics." 
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HEURISTICS OF MEDIA IMPLEI4ENTATI0N IN 
INSTRUCTIONAL SYSTEMS DEVELOPMENT 



Note: This discussion v?as adapted for use here 

from an address by Dr. John Haney, at the 
conclusion of the four development teams’ 
reports on Project activities at the 1967 
Convention of the Department of Audio- 
visual Instruction of the National 
Educational Association in Atlantic City, 
New Jersey. In several points, he refers 
to the preceding presentations by team 
leaders . 



The professors whom you saw in the preceding filmed interviews 
are real people, certainly, the kind we media specialists deal with 
every day. Remember the one who said, "I don't care about the 
development model," another who said.. "I don't care who reads the 
research," and another who said, "I'll be damned if I'll write my 
behavioral objectives"? Their remarks draw to mind an incident 
that took place a great many years ago and involved a new instructor 
at Columbia University. President Nicholas Butler said to him, 

"You know, some day you'll be looking out at a sea of workable faces, 
and somewhere in the back of the room you'll see a person who is 
frowning and shaking his head and you'll be tempted to reach out and 
grab him by the scruff of the neck and shake him. Don't do it, he's 
probably the only one that is listening." To a certain extent, these 
professors, whose remarks at times appear negative, are the ones 
working with us in this Project, and represent the many other instruc- 
tors with whom we media specialists will deal in our operations. 

As a media center practitioner, one whose hands are continually 
dirtied by the nitty-gritty chores of working with real faculty 
members, and real courses in a real environment, I am vitally interest- 
ed in the major purpose of the Instructional Systems Development Project; 
i.e., to increase the understanding of instructional development so 
we media specialists and instructional specialists can be more 
effective and efficient. 

For instance, my experience in this study tells me that in the 
future, whenever I deal with such a whopper word as "understanding," 
it should be further defined. Also, that as helpful as the con- 
struction of a hypothetical model or flowchart was in instructional 
development, and as valuable as its testing and demonstration are to 
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the results of this study, the most important product is more than 
simply a revision of these steps, or a refinement of the boxes and 
lines in the model. It is learning how to operate within the model 
to get desired results. 

hv t;hp bOX6S Or S t8PS ill 

the model, there is a '*how" or a collection of strategies, tactics, 
gambits, plays one devises or calls forth to make a model work. 
Obviously, the more of these things the instructors and specialists 
have in their grasp, the more likely it is they will be effective and 
efficient. However, these competencies are the mark of experience, 
not conflicting with formal preparations and theory in empirical re- 
search, but somehow parts of it. Often they guide us intuitively. 
Sometimes articulated and passed along by tradition like a ballad 
from one academic generation to another. These are the result of 
what one has found proceeding from new discoveries and from past 
discoveries which guide one’s course of action. 

The articulated heuristic may seem to be the poor cousin of 
the stated hypothesis of empirical research, but more likely it is 
the father. John Gordon, another member of the Evaluation Team, calls 
this kind of hypothesis a past heuristic, for which research is most 
profitable to the extent that heuristic is true. 

A principle, a functional principle, is a polished heuristic. 

Von Clauswitz, reflecting upon his experience in war, formulated a 
concept of surprise and concentration of force, a heuristic still 
used by victors and relearned by losers all the time. 

In this McCluhanistic age, heuristics are passed along, not only 
by word of mouth; this is, "if you can't lick them, join them," 
embroidered wall plaques, "a penny saved is a penny earned," and 
printed in books. Remember Dale Carnegie's best selling book of 
heuristics of the 30* s. H ow To Win Friends and Influence People ? 

He said, "Remember that to every person the sweetest sound in the 
English language is the sound of his own name." 

Heuristics appear on buttons that you see up and down the 
Boardwalk, bumper stickers, and TV screens. Heuristics are not the 
exclusive possession of the old and wise. As the young actor would 
say, "Never trust anybody over 30." Also, some distinction may be 
made between the rules of a chess game and the player's heuristics. 
The rules are stated -- they declare how the game is played. The 
heuristics are acquired — they are how the game is won. As Robert 
Davis, the learning specialist at Michigan State points out, "Nobody 
plays chess following a model because he'd get clobbered'." In the 
period after the opening formal moves, the player is guided by such 
heuristics as, "control the center of the board," or "develop each 
piece to the maximum." 
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What is the relation of heuristics to the kind of formal research 
we*ve been engaged in in this Project? Let us consider, as an an- 
alogy, the work of Kinsey, Masters, and Johnson as researchers in 
the field of human sexual behavior and response. These scholarly 
investigators presented their findings in statistical tables and 
analyzed the physiological process into a four- step flowchart. But 
as a guide to the practitioner in the field for strategic planning 
and the allocation of scarce resources, one is likely to rely more 
on the heuristic offered by Ogden Nash, *'Candy is dandy, but liquor 
is quicker.” 

In the case of the Project research at hand, there are a number 
of heuristics I have observed which relate the instruction«ii develop- 
ment model to reality, tie the steps to the real life process, and 
make it work. In addition, some of these heuristics generf^e re- 
search hypotheses, and form a major part of the content for a 
curriculum preparing instructional development specialists. Heuristics 
are not generally academically respectable, but these are obvious and 
may be useful because they are so often ignored. While admittedly 
over- generalized, they are, nevertheless, generally true. What they 
lack in rigor, they make up in vigor. For each of the heuristics I 
have chosen, I will cite an example from the Study. There are about 
fifteen I have selected for discussion here. 



Heuristic #1: The development of ” software” is more costly than the 

acquisition of ’’hardware .” How often do you see people fascinated by 
hardware? When you ask them how they're doing they may well reply, 
"Well, we're just beginning to get our hardware installed. We're 
going to have a two-week workshop to permit the instructors to write 
programs for the next year.** Programmed- instruct ion people some years 
ago had a rule-of-thumb-- 30 hours of program writing time for one 
hour of finished product. Production of validated materials re- 
quires a similar rule-of-thumb. The more often this need is ignored, 
the more often difficulty is encountered in multi-media, computer- 
assisted instruction, and in other kinds of instructional development. 

Heuristic The development of "software" is a continuous process . 
The development process here is not just a matter of completing a 
task and leaving it, but realizing that the production of, for in- 
stance, graphs involves a series of successive approximation. Even 
when the objectives are achieved, you may find that the instructor 
has altered his objectives. . This is more than ordinary evaluation 
or revision, it is a continuous process of refinement. 
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Heuristic #3: Always move toward finding the instructor's 

objectives . The model is explicit on where to start logically. 

But, what do you do when you run across a professor like the one 
who said, "I'll be damned if I'll sit down and write out behavioral 

" Tlr* vnii nnit*. dft von <»n to somehodv e^se? No. VOU 

really can't. But, you can have a bag of tricks at your command 
so when he states, "I think I want my students' curiosity wetted 
about this," you reply, "Alright, given you have a student whose 
curiosity has been wetted, what does he do?" This is one way of 
getting at behavior objectives, or as Syracuse pointed out, use an 
observation-verification approach; i.e., "Let's see your examin- 
ation and classroom. Let's observe what's going on. Let's 
articulate and state apparent objectives and see if you agree." 

This is one way of moving toward the objectives without having to 
bring everything to a halt if he doesn't want to sit down and define 
them. 

Heuristic #4: Involve the students continuously in the development 

process . You'll often find people in judging instructional situations 
saying, "I think," in speculating on the effectiveness of instruc- 
tional materials. The student, himself, is a prime source of 
information about what makes materials effective and useful for 
achieving his objectives. Often, significant revisions can be 
designed by having even a single student work through the materials 
and discuss the experience in an interview with the developer. 

Include the student in the development process whenever possible. 

Heuristic #5: The davelopmental model is universal only in a general 

way, functional clusters and linear sequence . We thought at the be- 
ginning of the study that we could test out the 15- step development 
process and come up with a more refined universal model. As you may 
have observed, each of the institutions introduced its own variation. 
Perhaps there is a sub-heuristic that says people never adopt, only 
adapt. This is certainly true here. There are some general 
character! tics in all of the kinds of models, not only in this 
study, but also in those that you've seen when anybody talks about 
"the systems approach to instruction." These models are all linear 
and they have a feedback loop and certain general functional clusters. 
The value of the model is to rationalize procedures. But also as 
Robert de Kieffer points out, it can be used as a road map to show 
you where you are in the development process, whenever dealing with 
an instructor. In this study, we've developed several points of 
entry calling not only for feedback, but also what we call "scoop- 
back"; that is, find out what the prior decisions were and other 
information before proceeding. 
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Heuristic #6: Stress the human elements in instructional systems « 

People have a stereotype about systems and technology, based 
usually upon systems analysis and systems application in industry 
and the military. Well, that doesn't really apply here. Our system 
allows for the human use of human being. We want to stress not 
only what the humans are doing, but what they can do as a reult 
of their involvement of instructional development. I would like 
to cite Blackwell's definition of technology -- the use of 
available resources to achieve value. As long as a person 
stresses that these decisions are based on human values, we 
have the proper relationship. 

At first glance, the instructional systems development model 
appears to demean the instructor. It places greater emphasis on 
learner involvement. It requires inputs from other specialists, 
involves mediation of instruction, extends his instruction, and 
forces him to reveal his pre- teaching decisions. But on the other 
hand, he has more attention paid to him, more recognition as a gate 
keeper. When he makes a decision things happen, and he is credited 
as being an innovator. Stress the human elements in the system 
operation, especially the students and instructor involved. 

Heuristic #7: Always proceed on the basis of agreement . This is 

important, especially when worMng with multiple sections and 
multiple instructor courses. Get agreement on procedures, criteria, 
objectives, and grading instruments whenever you can. Otherwise, 
there are too many uncertainties and uncontrolled variables. Now 
this does not always mean approval when we obtain agreement, but 
it reduces uncertainties and assures some commitment. 

Heuristic #8: Don't let the ”word*' or "manner” get in the way . 

"Manner" is the way of making a dogmatic statement that may cut 
off a faculty member. "Word" may simply be a part of our profes- 
sional jargon. We have to consider how statements affect certain 
faculty members. Sometimes an instructional development specialist 
may use the word "in format ion- over load," adopted from computer 
technology, and find that a humanities' instructor has just plain 
"turned him off." "Behavioral objectives" can also get in the way. 

I once had a bumper sticker in my office stating, "Help stamp out 
non- behavioral objectives." This pleased me a good deal. But, it 
didn't really help in the initial contact with insJ. actors. I 
don't even stick with the term "behavioral objectiT JS," if by using 
the term "specific objectives" we can obtain the same results. 



Heuristic #9: * *Blame** the professor first . Students do this all 

the time, so why shouldn't the development team? What is he like? 
Maybe we can find out something about the instructor, e.g. , what he 
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what does an instructor need here? We had a television instructor 
who used his graphic artist more as a nurse than as an artist. 

Media people can sometimes be the top professionals in what Eugene 
Oxhandier calls, "media therapy." This is business that we're 
sometimes pursuing in dealing with instructors. 

Heuristic #10; See that faculty members are rewarded for work in 
instructional development . The normal academic reward system is 
stacked against an instructor spending the required long hours and 
hard work in developing college courses. Find the effective and 
appropriate reinforcements for development work -- time, money, 
stratege, acclaim, support, etc., and crank these into the system 
and into the structure. 

Heuristic #11: Structure the conditions for survivability and 

transferability . How many times have you heard about campus teach- 
ing experiments, and by the time you get to see them, the thing's all 
over? It's gone, it's had it, and been dissipated out of existence. 
Why can't materials transfer from one institution to another? What 
is the unit that can be transferred? Perhaps not a whole course, 
but how might instructional design develop in one place and be 
effectively used and accepted in another place? How can we overcome 
this "not-developed-here syndrome" that results in virtual auto- 
matic rejection of materials produced at other schools? There are 
the things that have to be considered and built into the development 
process structure. For example, if you wish to introduce an 
instructor to a programming device, and he happens to be an English 
instructor, your natural tendency might be to obtain a program on 
grammar or punctuation, and let him play with it. Don't do it 
because he'll fight the first frame on the subject matter. Get a 
program on contract law, give him a case study there, and ask him 
to find out if it is a valid contract or not. This permits him to 
examine the characteristics and principles of the device without 
first debating the subject matter issue. Don't throw a contract law 
example to a lawyer -- throw him one on punctuation. 

Heuristic #12: When you reduce reality you also reduce the learn - 

ing experience . This is a very simple one, but it looks like Edgar 
Dale's cone upside-down. It simply means that as you move away 
from reality to simulation, you are also removing something from 
the experience itself. An aircraft simulation is not an airplane. 

A practice session or rehearsing before the music instructor is not 
a real recital. If the learning criterion involves elements, such 
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as audience presence, somehow the instructional system must in- 
troduce it. Don*t assume that you achieve more through simulation 
than is actually the case. 

Heuristic #13: Find the pattern or format that will balance 

benefits and liabilities . In bringing guests from the world of 
business into a televised course our institution offers in manage- 
ment, we try to strip the guest of all his "public relation" props 
and place him into an interview format, rather than a lecture 
format, in which he may read his public speech on the glories on 
the free enterprise system. But even with these changes, we find 
that the interview format can only be "saved" by the introduction 
and conclusion. Presently, we take an interview, review it, and 
then have the instructor insert into the tape an introduction 
telling the student what to look for. This pattern allows for the 
most effective use of the unique contributions a guest speaker can 
bring to the course. 

Heuristic #14; Faculty members are not generally moved to change 
behavior as a result of reading a report on instructional research . 
The reports that show no significant difference in improvement 
comparison studies produced no real significant difference in the 
behavior of the academic community. Occasionally, these reports 
may help prop up an instructor, if he has some misgivings. You can 
also sometimes use these as a way of countering a negative attitude. 
But, merely passing out the results of instructional research studies 
will not motivate faculty members to change. 

Heuristic #15; Nothing convinces like a visit . This is one on 
which part of this study was founded. But also, nothing can de- 
flate like a visit when you find out what is actually going on, 
especially from talks with students and other members of the 
faculty . 

These fifteen heuristics were drawn from the experiences of 
the development teams of the four project institutions. This 
collected wisdom shows not only what they have been doing, but 
offers some practical ideas of real use to media practitioners 
working in instructional development. 



USEFULNESS OF THE DEVELOPMENT MODEL FOR GENERATING 
RESEARCH HYPOTHESES AND SPECIFICATIONS 
FOR MEDIA SPECIALIST TRAINING 
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Research Hypothesis Generating Mechanism 

A hypothesis is a good hunch. A research hypothesis is a 
good hunch operationally defined. That is, it explicitly states 
both the present and desired states of affairs and the specific 
operations required to move from the former to the latter. In 
employing the described project model to design improved instruc- 
tional procedures and materials. It was hoped that the decision- 
making requirements placed on faculty members and development 
specialists would yield researchable problem statements. However, 
this evaluator found little evidence of such outcomes. 

What makes this explication so difficult? Is it not what all 
of us implicitly do when we generate an alternative solution to 
an existing problem? Is not what we choose to do out best hunch? 
If so, it would seem that a simple transcription of instructional 
decision-making should reveal a raft of hunches or hypotheses. 
These hypotheses should flow from the discussion as each par- 
ticipant offers his favorite alternative. They don't. Here are 
some reasons why this writer believes that research hypotheses do 
not automatically "flow" from practical decision-making. 

1, Perhaps the most obvious reason is the failure or in- 
ability of instructors and development specialists to 
make explicit the present and desired states of affairs. 
Alternatives are generated and selected with little 
concern for the exit behavior of the students (desired 
state) and even less for the entry behaviors (present 
state). Instructional strategies are usually deter- 
mined either by the instructor or specialists according 
to their bias for one of the newer instructional tech- 
nologies; i.e., CCTV, programmed instruction, etc., 
situation. The tedious specification of behaviors at 
a level needed for acceptable research hypothesis 
statements is often by-passed to get to the urgent 
production of demonstrable packages in time for the 
meeting of classes. In other words, the necessary time 
needed for course and task analysis is usually greatly 
underestimated with the result that production is 
seemingly always ahead of the objectives. 
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2. The preceding rationale hints at the second reason; 
the specificity of the situation. The instance, the 
chosen alternative, stands by itself. It is not 
general to a class of situations and, therefore, 
obviously not meeting another important character- 
istic, generality. Even if a general hunch is 
applied to the specific situation, it is not readily ap- 
parent from the decision made. It is in the decider's 
head, and perhaps even unknown to him. It is extremely 
difficult to go from the specific "back to" the general, 
if indeed, it was there to begin with. In the observed 
development activities, no one usually broached the 
question of generality and the faculty members and 
specialist moved on to the next decision. 

3. The third reason was also hinted at in the first dis- 
cussion; decision-making in a restricted environment. 

For example, it is difficult to imagine a more limitating 
situation than one in which a faculty member is assigned 
to teach in a given room at a particular scheduled time. 
What might-have-been becomes the never-even-considered. 

For example, teaching method alternatives must be gen- 
erated from the stale, traditional stockpile of what is 
there, rather than more general applications of feedback 
capabilities or response environment. 

4. The fourth reason speaks to another facet of the general 
problem and perhaps is simply another way of saying what 
came before. It centers on the peculiar characteristic of 
instructional alternatives -- the trade-off. The teaching 
situations not only suffer restrictions of a physical sort, 
but of a personal value sort as well. Another way of 
putting this is that each of the participants in the de- 
velopment process has a number of "hidden agendas." The 
description of the present, desired, and operational states 
in a classic research hypothesis carries only implicit 
values. A business decision, for example, is made when all 
the relevant outcomes and operations have been measured 

by either their market or personal service value. However, 
the values of decisions in development procedures are 
weighed in a number of ways, still unknown to the math 
modelers. Some decisions reached may be directly contrary 
to the trial of the best hunch simply because operational- 
izing the best hunch "costs" more than the decision-maker 
can afford, either personally or financially. 
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The last point challenges the relevancy and usefullness of 
the traditional nonutility research hypothesis in the practical 
decision-making process. It should be replaced by value-explicit 
predictions using operational research, decision and Bayesian 
Theory statistics and methodology to take account of risks and 
regrets. 

The regret and risk of these observations are that (1) ve 
regret to inform the curriculum designers for media specialists 
that they will have to consider a radical departure from the old 
Fisherian notions of design, statistics, and decision-making, and 
(2) we risk our reputations on this protestation for we really 
don*t know enough about it to take such a dogmatic stand. 

We are relatively sure that the media research, thus far, 
has not come predominantly from media specialists and the media 
research, that has been done, is not readily translatable into 
practical day-to-day operations. 

That hackeneyed phrase of "being all right for the theory 
but not good for practice" may be trying to tell us that the real 
world is not valueless for research, that development decisions 
are actually tradeoffs, not simple statements about "significantly 
greater learning." 



Implications for Media Specialist Professional Preparation 



Questions raised as to the competency breadth and depth re- 
quired by the typical media specialist are complicated by the fact 
that he usually must be both "consultant" and "doer. These con- 
cerns refer specifically to the knowledge and competency in 
development tactics he should possess. For example, should a 
media specialist know the principle of test item development, 
instructional conditions specification, behavioral objectives 
definition, etc.? Where does the media specialist begin with the 
professor-client in the overall scheme of instructional systems 
development? How much is he expected to know, and how many 
capabilities should he have? Presently a major responsibility 
has befallen the media specialist. As the concern for better 
instruction has grown and more questions asked about the 
appropriateness of methods, the media specialist has been forced 
to offer some answers, even though his better judgment sometimes 
suggests he should not. The current concern for rational and ex- 
pert instructional development practices has revealed the great 
complexity of instruction, and bared the large areas where pure 
conjucture is the best we have to offer. The media specialist. 
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in trying to be all men to all people, has laid himself bare to 
the accusations of those who expect direct answers rather than 
hinted possibilities and statements of probabilities. Where does 
his competency begin and extend to? Can he really become a 
specialist, or must he be forced to remain a generalist about the 
complexities of instruction, and be considered only a first-rate 
t inker er and mechanic? 

This project has attempted to underscore the need for other 
specialist assistance. For example, the evaluation and instruc- 
tional specialists. Also, it proposes that the media specialist 
should be allowed to confine his expertise to a much more 
selective, nonetheless, complicated arena. 

What then is the specific domain of the media specialist, and 
how do we go about preparing him for this position? 

The curriculum or learning experience for preparing the media 
specialist should include two elements based on the observations 
made in this project. The first set of experiences should be ex- 
tensive demonstrations of: 

1. The attributes and functions of the machines and materials 
with which the media specialist works; e.g., the "boiling, 
burning, and elasticity points" of materials, the stress 
and load that can be expected from machines. 

2. The attributes and qualities of man as a message sender, 
receiver, and decision-maker; e.g., the rate, pitch, 
and loudness at which sounds become intelligible, the 
immediate storage capacity of a child; i.e., how long 

a sentence can he repeat, as in the Stanford-binet 
Intelligence Test. 

3. The fundamental operations or interplay between man and 
machine-materials which cause a change in one or the 
other; e.g., how to operate a projector, or how to 
create maximal conditions for man learning a multiple 
discrimination. 

The second set of experiences would extensively involve the 
media specialist trainee directly in the practice of point three 
above under both simulated and real conditions. Two separate, 
but obviously related activities, should be practiced: 

The communications interplay between client and specialist, 
during which decisions are made in the instructional design 
process . 
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2. The actual implementation of the specifications outlined 
by the decisions reach( i in the course development steps. 

The dialogue between specialist and instructor- client 
employs the knowledge of tnan as a decision-makers The 
accompanying facilitative operation is analygous to the bed- 
side manner of the physician. It sometimes can be likened to 
a clinical interview or even a detective’s interrogation, as 
when certain principles or tactics are brought into play to 
tease out certain information, such as behavioral objectives. 

It is precisely these tactics and practice in them that is dis- 
regarded in current development programs. Equally disconcerting 
is the fact that the second phase, the actual carrying out of the 
decisions, can often be done by another specialist — perhaps in 
graphics. A change in role, i.e., from a doer, maker and repair- 
man, to a consultant who works with people and their ideas and 
then instructs another who does, makes, and repairs, marks the 
great needed transition for media specialists both in prep- 
aration emphasis and professional practice. Despite this 
assertion, it is likely that in many institutional settings the 
media specialist must continue to both be consultant and doer 
for some time to come. For this reason, the training programs 
should also include the second set of practice experiences, as 
well as those dealing directly with the materials and machines. 

The specialist, in acquiring some skill as passable graphics 
artist, photographer, and equipment maintainance is then better 
prepared to serve under varying conditions. 

1. He can assure the materials development will meet some 
level of acceptability. 

2. He becomes conversant with the "trade," so as to speak, 
and can communicate with a more skilled artisan or more 
specialist in the dimensions of the task, where they are 
available to assist. 

But back to consulting role, lest we omit discussion of what 
is really quite obvious, that all three aforementioned sets of 
training principles are interdependent. Of the three, the tactics 
used in working with the client are the most critical set of 
principles. The other principles all come into play when con- 
sidering decision alternatives; e.g., how much light is needed, 
for how many people, for what specific conditions, etc. , and 
calls forth the total repertoire of the consultant. 
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In summary, two related, but separable areas of competency- 
should be in the repertoire of media specialists. The first 
set deals with designing overall information transmission 
logistics. That is, given the major facilities, time, personnel, 
and financial limits, v;hat major alterations are needed to ac- 
commodate strategies and materials of instruction if they are 
to be considered alternatives? 

This advisary capacity is not to be misconstrued with 
deciding finally on what should be specifically done. It is 
more a listing with reservations of the possible system 
capabilities. These alternative capabilities can be, and many 
times are, specified before any consideration is given the goals 
of a particular course or objectives within the courses. The 
result is that often the "efficienty" of the means of instruction 
influences selection decisions more than "effectiveness," which 
is a more goal-oriented criterion. 

The second area of competency deals with the specific 
message to be transmitted. The objectives should be well-spelled- 
out and the specific examples or information to constitute the 
message selected. The representative or media of the message and 
the learners human sensory modalities must be considered along 
with the professor’s strengths in communication. The represent- 
ative form decision centers about the capability of the learner 
for acquiring referential meaning from the message. If he has 
never seen an auk before, and the attributes of an auk are im- 
portant, then some attempt to see an actual auk or meaningful 
representation seems imperative. The referent model can be 
selected from symbolic, iconic, analagous, or inactive categories, 
depending upon a number of criteria. Principles relating to 
size, color, contrast, etc., in iconic representation alone fill 
a number of books. Beyond need for a reasonable amount of 
knowledge in such a subject, the media specialist could well 
afford to rely upon the judgment of the artist. The physical 
form alternatives of the message, must be measured against the 
learner sensory modality competencies and limitations. For 
example, the non-reader faces some limitations in the rate and 
intensity he can receive audio direction. The critical question 
as to whether one can receive two separate messages simultaneously 
in two different modalities still plagues media specialists. Some 
feel that slow transmission of two redundant messages is not 
harmful. Whether it is more effective than single source and 
message is still in doubt. 

This selection of form and modality is extremely complicated 
and coupled with both interrogation tactics, overall logistics, and 
a modicum of manipulative skills is enough speciality for one man, 
perhaps even too much. Assuredly, it soon will be, as research 
broadens the knowledge and skill in these areas. 
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DISSEMINATION VALUE OF THE DEMONSTRATION APPROACH 



Dr. John Childs 



Introduction 



The initial proposal for this project defined a three- fold task 
for the project staff. Two of the tasks were: ''Disseminating the 

model developmental system," and "Evaluating the dissemination 
activities." The techniques used to approach these tasks were derived 
from innovation research reported by Brickell, Rogers, and Miles, et al. 
The basic design was that suggested by Brickell in the statement: "The 

most effective way to convince a school staff that it should adopt a 
new program is to let it observe the successful new program in action. 
Nothing persuades like a visit." 

Under the foregoing assumptions the project staff launched two 
basic approaches to disseminating the model developmental system: 

First, the training of institutional demonstration teams; (this in 
itself constituted a form of diffusion). Second, the process of having 
the demonstration teams work within their own institutions on the 
application of the model developmental system. 

In the reports from the demonstration institutions, tv/o types of 
dissemination were evidenced. First, dissemination of the model devel- 
opmental systems occured within the institutions' colleges, departments, 
and courses. Second, visitors from other institutions communicated 
V7ith the demonstration teams about the application of the model 
developmental system. 

Definitions 



The terms used in this report to delineate the process being 
discussed need to be formally defined in order to avoid confusion. 
Such formal statements follow: 

1. Dissemination - the process whereby an innovation 
comes to be shared. 

Dissemination - process of spreading innovations 
into schools. 

2. Diffusion - the process by which an innovation spreads. 
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3. 



Innovation - "an idea perceived as new by the 
individual (s) . " 

Innovation - "a deliberate novel specific change, 
which is thought to be more efficacious in ac- 
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of specificity might more easily be labeled 
' thingy-ness.’ " 

4. Adoption - a decision to continue full use of an 
innovation. The adoption process is the mental 
process through which an individual passes from 
first hearing about an innovation to final 
adoption. 

Adoption of innovations - "(1) acceptance, (2) 
overtime, (3) of some specific item — an idea or 
practice, (4) by individuals, groups, or other 
adopting units, linked to (5) specific channels 
of communications, (6) to a social structure, and 
(7) to a given system of values or culture." 



Evaluating the Usefullness of Demonstrations as a Dissemination Method 

The preceding formal definitions leave a considerable amount of 
room for interpretation. There is some inaccuracy in the application 
of these terms in this document. The user of the innovation concept 
to account for change in the educational institution .frequently 
applies the terms "diffusion" and "dissemination" interchangably* 

The two words often take the place of "innovation," 

The definition given above for "adoption of innovation" is more 
precise. In the current study the third through seventh sections of 
the definition are actively pursued. Emphasis in this segment of the 
report focuses on the individuals and groups, the communications 
channels, and the social structures. The specific item(s) being 
"diffused" or "disseminated" is (are) the model of instructional 
systems development. In essence, the "process" is the item. Under 
such circumstances the traditional use of terms applied to the 
"adoption of an innovation" becomes difficult. A process cone.- '‘ually 
may have "itemness" characteristics, but when the communicati''’" f the 
process is attempted, the characteristic of itemness is diff iv. -xt to 
identify and define. In the case of this project, the most tangible, 
item- like representation of the innovation being disseminated, is the 
symbolic model of the developmental procedures in the design of 
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instructional systems. The person who turns to the project for a 
tangible product, when asking what was disseminated, will be grossly 
misled and probably disappointed with the tangible representation 
of the innovation — the flowchart of procedures. 

The distinction made above between an attempt to disseminate 
a "process," rather than a "product," is most important. The bulk 
of previous research on "adoption of innovations" has focuses on 
product innovations. While a product- oriented model was selected 
for the basic strategy, the item being disseminated turned out to 
be a "process." The techniques for disseminating "process" in- 
novations in a demonstration mode were not defined in advance of 
actual operation. Thus, much "feeling around" took place in the 
communications process to determine methods for expressing the 
substance of the innovation within the demonstration institutions. 
The actual dissemination attempts within institutions became highly 
personal, individualistic, and display much variance. 

In addition to the dissemination activity within the demon- 
stration institutions, mass efforts were conducted at national 
conventions and by publications from the coordinating unit. 
Assessment of these activities was limited. However, the volume 
of correspondence requesting information indicated that much 
information concerning the existence of the project, and its 
efforts, was transmitted through the papers present at the various 
conventions and by way of mail correspondence. The coordinating 
unit retrieved from its files 88 letters of request, based on the 
1965 AERA presentation and on the 1965 and 1966 DAVI presentations. 
In general, these letters requested copies of the various papers. 
Some asked for more specialized information concerning application 
of the procedures in the systems development model. 

The dissemination activities within the individual institutions 
participating in the study were tapped by survey questionnaires, 
direct interviews, and personal reports of the participating staff. 
The data are summarized on the following pages. 

The instruments used in an attempt to keep track of the various 
dissemination activities in this project produced only nominal data. 
As the project developed, increasing emphasis was placed on the 
further development, refinement, and assessment of the validity and 
reliability of the systems development model. This led to a reduced 
emphasis on the collection and study of data relevant to the 
"adoption of innovation;" consequently, assessment of the results of 
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dissemination activities in terms of specific adopters were 
impossible. Available data were limited to the personal view- 
points of the participants and the selective perceptions of the 
project evaluators and the advisory committee. The ’*who, what, 
when and where” descriptive data collected tended to indicate a 
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college and university personnel making intensified efforts to 
imprpve instruction. 



From the beginning, some very basic disseminati a data were 
lost to the dissemination evaluator by the very nature of the 
project. Those initial contacts between project staff members 
and potential adopters necessarily were unrecorded in many 
cases. The interest of the demonstration teams in the refine- 
ment of the model also reduced the amount of effort devoted to 
the dissemination phase. This would be expected whenever a 
given innovation had not reached the initial acceptance stage 
where it could be applied without major alteration. The "process" 
nature of this particular innovation operated to decrease the 
probability that the innovation would be applied in its dis- 
seminated form. Thus, it became difficult to trace dissemination 
events to actual adoptions. 



It would be hazardous to generalize basic dissemination data 
for this type of "process" item from this study. The variables 
operating in each demonstration situation were largely undefined. 
Yet, it appears quite evident that they were extremely different 
for each situation. One might find it possible to generalize the 
model to other situations, but the dissemination procedures remain 
largely untested as to their validity or reliability in other 
applications. 

Some major pitfalls might be held up for consideration in 
other situations where the innovation is a "process" rather than 
a'{>roduct." First, some basic understanding of the process must 
be a part of the demonstration team’s repertoire before attempts 
are made to communicate the "process" to other potential adopters. 
Second, evaluative and refinement efforts should not be a part of 
the dissemination effort. Probably both should precede the dem- 
onstration phase. While this project adopted the demonstration 
design in order to facilitate the communication of the model, 
Brickell’s further comments about the conditions of the demon- 
stration were largely ignored. Brickell points out that 
"anything which the observer could lable ’abnormal* or 
'unrealistic,' such as the enriched conditions necessary for 
good design or the controlled conditions necessary for proper 
evaluation, is sufficient to rob the observed program of 
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persuasive effect," In this project, dissemination was hampered 
by the need to refine and evaluate the model, by the lack of a 
"normal" environment for the application of the "process" (time 
pressures, extra funding, etc.) and by the absence of adequate 
training for the demonstration teams. 

In summary, it appears that a great deal of dissemination 
occureed in this project of a "process" systems development 
model. The dissemination occurred through personal contacts, 
demonstration center visits, national convention papers, and 
published articles. Presentation of the tabulations of these 
events for each demonstration center follows. 



SUMMARY OF DISSEMINATION INFORIiATION REPORTED BY THE UNIVERSITY OF 

COLORADO 

No. of courses to which the procedures of the project were applied ; 
Four 

No. of faculty persons involved in the project ; 

Ten 

No, of refusals of the procedures by individual faculty members ; 

One 

Communications concerning the effect and future use of the 
project procedures ; 

On January 19, 1967, a conference was held to communicate 
information about the project and to interest staff 
members of the university in the application of the 
procedures. 

Eighteen faculty representatives attended this con- 
ference. 

Twenty- four guests from other institutions attended this 
conference . 

The procedures of the project were presented to the spring 
meeting of the Colorado Audio-Visual Association. 

Many additional informal discussions were held to interest 
persons in the systems procedures. These included: Boulder 

Valley School Administrators (initial reaction was that they 
would apply the system); Airforce Academy; WITCH; and WITCHE. 



Special documents used in dissemination of the project procedures : 
Instructional Systems Development brochure produced at 
Michigan State University by the co-ordinating unit (green). 

Brief dittoed summary of the project at the University 
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Project document labeled "Aid to Identification of Course 
Objectives: Part I," 

Proposal document for the assessment of adoption behavior 
with regard to the model development system (prepared by 
John Childs). 

Brief summary material prepared from the Taxonomy of 
Educational Objectives - Handbook Cognitive Domain 
(Benjamin S. Bloom, Editor). 

Brief summary of material prepared from Preparing Objectives 
for Programmed Instruction (Robert F. Mager) . 

A slide, audio- tape, video-tape presentation was prepared 
for the vTanuary 19 meeting on the campus. A copy was 
reviewed vat Michigan State University. 

Visitors to the Demonstration Center ; 

Those representatives named above as attendees at the 
January 19, 1967 Conference. 

Selected comments of visitors ; 

"The idea is sound." "We would hope to establish such a 
system." "An organized means of improving instruction is 
definitely needed." 

Selected comments of the demonstration team as repotted on 
January 26, 1967 ; 

"The organization of the team (Media; Evaluation; Learning) 
is appropriate and should operate as a team," 

"Three to five years is needed for evaluation and recycling 
of a given course." 

"To be effective, there is a definite need for released time 
for the faculty members involved, as well as the team members." 

"The system can be adopted to all levels of education and all 
subject areas." 
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SAN FRANCISCO STATE COLLEGE 






No, of courses to which the procedures of the project were applied : 
Two 

N o, of faculty persons involved in the pifojecu ; 

??? 

• • • 

No, of courses to which the procedures were applied outside of the 
project : 

Two 

No, of faculty persons using the procedures outside of the project : 

??? 

• # # 

Communications concerning the effect and future use of the project 
procedures : 

Reactions to a number of individual requests for information. 

Presentation to the faculty of the School of Education, 
History Department, Psychology Department, and Economics 
Department. 

Special seminar for the School of Education. 

Special seminar on systems economics which included rep- 
resentatives from local school districts, San Jose State 
College, and Foothill College. This seminar x^as held off- 
campus. 

Selected comments of visitors : 

"Specific curriculum revisions are occurring as a direct 
result of the project." 



\ 



! 



SYRACUSE U14IVERSITY 

No. of courses to which the procedures of the project were applied ; 
Two 

No. of faculty persons involved in the project ; 

Two 
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No, of faculty persons using the procedures outside of the project ; 
Eight 

Communications concerning the effect and future use of the project 

^ ^ ^ ^ m 

pi, W^CVIMJL go » 

Mr. T. E. Bur ford reviewed the Instructional Systems 
Development Project, its nature and progress, with Dr. 

Taher Razik, Director of the Audiovisual Department, 

SUNY at Buffalo. 

Mr. C. M. Koch presented a lecture on the Instructional 
Systems Development Project at the New York State Ed- 
ucational Communications Council Convention in November, 

1965. 

Mr. Koch, now director of Learning Resources, SUNY at 
Cortland, is now using the project procedures in that 
institution. 

A one-day conference was held at Syracuse on March 10, 

1967. Invited participants were: Deans and/or Vice 

Presidents for Academic Affairs at New York State Schools 
of Education and their audiovisual people. Approximately 
60 representatives of 28 institutions attended. 

A special presentation was made for representatives from 
Cornell University, Ithaca, New York. Nine persons were 
included in this group. 

Special documents used in dissemination of the project procedures : 
One-day conference program. 

Four- page, 8-1/2 x 11, printed brochure with photographs 
entitled: Instructional Systems Development Project , 

Center for Instructional Communications , Syracuse 
University . 

Kinescope of teaching activity in specialized classroom, 
using resources and techniques developed through the 
procjduresj with commentary by the subject instructor, 
as to the effect of the system procedures of the project. 

Visitors to the Demonstration Center : 

Those named above as attendees at the one-day Conference, 
plus those from Cornall who attended the special 
presentation. 
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Selected comments of visitors ; 

•'Very interested in systematic approach to course 
development." 

"Interested in development and utilization of multi- 
media classrooms, both hardware and software,” 

"Interested in applications and utilization of com- 
puter-responder system." 

"General interest in the heuristics of course development 
activities," 

"My primary concern is for cost analysis of development 
activities." 

"Professors have little time for planning classes or 
stating behavioral objectives. We know that the media 
can make the job more effective and more efficient." 
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MICHIGAN STATE UNIVERSITY 



Presentations of activities were conducted by the project 
demonstration and coordination units in: 

a. USOE Demonstration Center 

b. State conventions (2) of the Michigan Audiovisual 

Association 

c. Staff of the College of Human Medicine, M,S.U« 

d. MSU, Title VIB Media Institute Program in Higher 

Education 

e. MSU, NDEA Title XI Institute for College Media 

Center Directors 

f. Faculty Members at Indiana University 

g. Media Institute Participants at Teachers College, 

Columbia University 

h. National Conventions (2) of the Oepartae^^i of 

Audiovisual Instruction 

i. Faculty members at the University of Connecticut 

j. Faculty members at the University of Illinois at 

Chicago 

k. Wayne State University, NDEA Title XI Institute 

on Instructional Systems Development 
!• MSU, Special Media Institutes (5) for Directors 
of NDEA Title XI Institutes for Advanced Study 
in Geography and Economics 

m. Articles in the periodical Audiovisual Instruction, 

Department of Audiovisual "Instruction, National 
Education Assn, 

n. National Audiovisual Leadership Conference at the 
University of Iowa Lakeside Laboratory 

o. National meeting of American Association for 

Educational Research 

p. Convention of the National Association of 

Educational Broadcasters 

The response co these presentations was encouraging. 
Participants indicated considerable Interest in obtaining copies 
of the summary report depicting the outcomes at all four instit- 
utions, Correspondence indicates that observers referred other 
parties, not attending the presentation, to make inquitiws also. 
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COST ANALYSIS OF DEVELOPMENT PROCEDURES TRIALS 

Despite the fact that this study was aimed primarily 
at obtaining data on the efficacy of certain instructional 
development procedures, inevitably someone is bound to raise 
the question concerning the costs involvedo The investigators 
observed in the 1963-65 study at Michigan State, that the bulk 
of expenditures were for the costs of specialist and faculty 
member time to carry on development activities. The expense 
of graphics, photographic supplies, videotape and other newer 
media supplies were small by comparison. This same relationship 
of costs appears to hold true in the four- institution trial of 
the development procedures. All participants were made aware, 
before joining in the project, that the funds provided by the 
United States Office of Education would be insufficient to cover 
complete personnel costs, including faculty release time and 
staff specialists. Project funds were budgeted to insure the 
purchase of the raw materials for media production and the use 
of associated technical services to fulfill the decisions 
reached in the development deliberations. 

In an attempt to account for the overall investment in 
development time and dollars for materials needed to carry out 
the commitment in the analysis and revision of the two selected 
courses, a cost and time estimate for faculty, specialists, 
technicians and materials were submitted by the demonstration 
institutions. These data include expenditures covered by the 
project grant and in addition time and materials donated by 
individuals and the institutions. 

It must be emphasized that the data reported here was not 
collected through exact accounting procedures, and represent 
primarily gross estimates of materials and effort entering Into 
the course development efforts. 

This estimate summary covers costs and time expenditures 
over a two year period for development in two courses of instruc- 
tion that each enroll a minimum of 500 students per year. 



1, Faculty member time for development 
deliberations 

2. Instructional Specialists, Media 
Specialists, Evaluation Specialist time 
for deliberations and media development 
activities, 

3, Media production technician time 

4. Expenditures for materials and 
equipment rental 



800 - 850 hours 



3500 - 5000 hours 



800 - 900 hours 



$10,000 - $13,000 
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It is observed that development time easily outdistance 
the costs of materials in the instructional development process. 
The experience in this project suggest a 4:1 ratio exists be- 
tween the expenditures for development procedures and the costs 
of producing and implementing the newer media. Those contemp- 
lating renovation of course instruction would do well to allow 
sufficient resources for development work as insurance that 
their ultimate choices of teaching methods and materials are 
the most effective available. 



CHAPTER IV: 



DISCUSSION 



The instructional development steps examined by the 
specialist teams and demonstrated for educators visiting 
their institutions cannot be considered entirely novel. 

The skills underlying decision steps are the products of 
experience and research in the fields of learning, evaluation, 
and instructional media. 

At the start of this project, the demonstration specialists 
and workers at other institutions, already possessed some of 
these knowledge and skills and had applied them in the past to 
instructional design problems encountered in professional assign- 
ments. 



The innovative aspect of the project was teaming special- 
ist capabilities to the decision process of the instructor and 
bringing them to bear on instructional problems under the guid- 
ance of a preconceived, sequential system of decision-making. 

The actual worth of any such plan can largely be judged 
by its ability to observe the prerogatives of the academic 
community, and yet encourage the soul-searching necessary for 
identifying needed changes in methods and materials. This 
chapter presents some key observations in this respect offered 
by project evaluation, development specialists and faculty 
members at the periodic Project Review Meetings. 

OBSERVATIONS OF AN EVALUATOR ON THE PERSONAL PERFORMANCE 
OF PARTICIPANTS EMPLOYING THE INSTRUCTIONAL DEVELOPMENT MODEL 



1 . Difficulty in dealing directly with behavior objectives ; 

The Ic^licatipn is that in the flowchart the item 
"Specify Entry and Terminal Behaviors" comes in a step which 
is very early in the development process. Thus, if there is 
difficulty in this specification, either the flow stops, or 
there should be some way for running through the system at a 
low level of approximation and then cycling back to re-establish 
a better standard in this step and re-run the system successfully 
to upgrade the specifications it produces. 

2. The relative absence in some instructors of a repertoire 
of evaluative rationale (testing and measuring) and 
evaluative instruments. 



The implication here is that in the flowchart there is the 



Iddatitf feat ion of an evaluation specialist in the goals and 
objectives identification step; however, in some of the tapes 
there was little mention of this aspect, and the efforts to 
meet with the department chairman or coordinator of subject 
matter gave enough evidence to suggest that it would be hope** 
less to have started on this point of determining the evaluat- 
ive devices and instruments. For example, it seemed to this 

jLi.bU«fuexr uiiau i-iie uuly Cx.xi.ccxOtl uiiau uut: uuc uii£ii»&uuiu ujl uu^ 

department would have injected would be that, whatever other 
people asked for, his standards would have to be ten percent 
more difficult. 



3. The limited perception of the instructional media and 
the role of media specialist ; 

Limiting these to the simple task of making or trans- 
mitting, BUT NOT INTRUDING EITHER ON THE SELECTION OF THE 
MESSAGE NOR THE EVALUATION OF THE MESSAGE. For example, in 
the tape dealing with course A, it was very evident that the 
instructor and the whole activity was geared to a concept of 
feedback and all the instruments of any type were acceptable 
for that purpose; but in course B there was no such intergrated 
approach, even though the coordinator and the media specialist 
were continuously injecting the invitation to become more in- 
volved in the evaluative feedback activities. In the course 
B development team, the media specialist had to buck against 
a very limited perception of media function and of role of 
media specialist. 

4. The differences in the application of the development 
model that are dependent uF^n the subject matter. 



In some tapes, there are illustrations between course A 
and course B; and more specifically between the course B as 
general education and course B in a professional sequence. 
Obviously, all of these are subject to interpretation by the 
instructor, but it is such interpretation that changes the 
nature of the subject ^matter. Some subject matter is more in 
a verbal domain, some more in visual domain, some more in audio 
domain. Some is structured in relationship of "if-then” 
relationships, some open-ended to produce a "gee-whiz" effect. 



5. Differences in instructional styles and in the learning 
styles of students . 

It seemed very significant to this listener that course A 
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even among his colleagues, as a person who thought visually and 
who used visual metaphors in his Instruction. It was extremely 
easy to follow this instructor and the instructional development 
team in their conversation both as to what is to be accomplished 
in the course, the behavioral outcomes and the relative level 
of performance of these behaviors, and strategies and instrumen- 
tation for the desired instruction. By contrast, the instructor 
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o£ course B seemed quite Incapable of illustrating exactly 
the points he was making. Strangely enough, to me it seemed 
that this latter person was more dependent upon concrete 
instructional materials than he realized and much more so than 
the other instructor -- who undoubtedly used more and a greater 
variety. Specifically, the course B instructor impressed this 
listener as being the kind of a person who is dependent upon a 
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study-recite-test organization -- the role of the live instruc- 
tor being that of making the assignments tougher and pushing 
the students more intensively and rapidly along the established 
continum. But, there is little evidence that he sees this depen* 
dence nor that he wants to understand the medium and the learning 
styles the medium imposes on the student and the instructor. 



Note: The implication for the development system model is that 

it does not take into account either the concept of 
levels or the concept of different strategies either for 
the instructor or for the learners. By developing the 
model as an illustration applied to any module or bit of 
a larger instructional sequence, this difficulty could 
be overcome in applying the model to the whole job of 
planning a course. 

6, Personality differences and ego involvements of the 
instructors. 



Some instructors could ruin this system or this system 
could ruin them. The implication for the chart is that there 
probably should be some kind of a no-go check point that screens 
out something that shouldn't be thrown into this system. In the 
summary to the questions following a development conference, one 
of the participants indicated that he was pleased with the pros- 
pect of not having any further dealings with another participant. 
This is an excellent insight revealed from a normally hidden 
agenda; and it suggests provision in the flowchart model to kick 
out raw materials that cannot be ingested into the system with- 
out damaging it. 

REACTION OF A FACULTY MEMBER TO DEVELOPMENT PROCESS EXPERIENCE 

"Having played two rol4s in the course development 
project, it is extremely difficult for me to assess the value 
of the project from either point of view. One of >^he reasons 
for this is my feeling that IF I had not been involved as a 
member of the committee, I probably would not have stayed with 
the project as a faculty guinea pig. There were times when the 
committee challenged my choice of goals and methods, and in so 
doing, forced me into the uncomfortable position of admitting 
that I might be wrong in a decision made so long ago it had 
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been forgotten. Having stayed through once cycle of the project, 
though, 1 can say that the things that made me so uncomfortable 
were the things that needed doing In the worst way. In short, 
participation has been a painful but valuable experience — • 
rather like the army. 

One of the major weaknesses of the project seems to be 
Its obviousness. That Is, I felt (and others have Indicated 
the same feeling) that the ^'systems approach’^ didn't tell me 
anything I didn't already know. After you've seen the system, 
there are just two things you can say; "Of course," and "So 
what?" To put It another way, the project (as I understand It) 
seems to be testing the hypothesis that a planned course is 
more effective than an unplanned course. This hypothesis 
Is "obvi'^'^sly true" and at the same time "demonstrably false". 

That Is, It would be a rare event Indeed if a totally unplanned 
and disorganized course succeeded at anything, and on the other 
hand, planned nonsense Is just as nonsensical as unplanned 
nonsense. 

My conclusion is that the chief value of the project 
does not lie in the formalized system. Good teachers plan 
without It, and bad teachers can be bad In spite of it. Rather, 
the project's value derives from the "fact" that group decis- 
ions, under conditions of high uncertainty, tend to be better 
than decisions made by an individual. It is the "head bumping" 
that counts rather than the order in which it is done, IT 
HELPS WHEN THE "HEADS" DO NOT ALL HAVE THE SAME BIASES, 

A second value of the project, for me as an instructor, 
was that it provided funds to implement certain media decisions 
that could not have been implemented otherwise. Money, of course, 
can come from any source, btt the project has served to demon- 
strate that a relatively small amount of money can open doors that 
normally stay closed — can make possible better teaching, (I 
might also add that a small sum can do more for a teacher's 
morale in a "materials budget" than added to his salary. Large 
sums should, of course, be added to his salary.) I must also 
add that all the media experiments FAILED: the special rating 

sheets turned out to be unmanageable; the overhead materials 
were invisible; and the video taping of speeches almost sunk the 
ship (attendance dropped by almost 50% and isn't back to normal 
yet) BUT we learned from these experiences what we could not 
have learned any other way. Having made these "blunders" in an 
experimental stage, we were not so committed that, psychologically, 
we were forced to deny the failure, or, administratively, forced 
to repeat the "blunders" to justify having made them in the first 
place. 

A third value of the project has been that it has given 
me the self-confidence and the authority to introduce a new course. 
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Yesterday (April 19) my department approved the offering of a 
new course modeled after the course which we worked on in the 
project. Sure, I*d had the idea for a long time, but not much 
had been done with it. X^orking In this project, the idea was 
refined and polished. I gained confidence from talking it 
through with the committee. My colleagues, knowing that the 
idea had been through the mill, were more willing to give it a 
try. NOW WE HAVE TO 14AKE IT WORKl 1 1 

Hy major regret is that the project facilities (including 
the people) may not be available when I need them most — • next 
year. 

OPINIONS OF PROJECT DEMONSTRATION TEAM MEMBERS REGARDING 
RELATED ASPECTS OF INSTRUCTIONAL DEVELOPMENT 



In the process of conducting the Project a number of 
new questions arose which were not a part of the original plan 
of investigation, but on which the demonstration teams contrib- 
uted opinions. These responses are reported here under the 
general statement of the question asked. 

What conditions best augur the ccnmnencement of Instructional 
development procedures? 

Answers: 

1. A committment by the institution's administration for 
funding and faculty release time. 

2. Approval and support of the college dean and deparment 
chairman. 

3. The course is a required course or "basic course." 

4. Willingness of the instructor to experiment with change. 

5. Recognition that the course is in bad shape. 

6. Availability of development time for both the instructor 
and specialist staff meinbers. 

What factors contribute to the continued use of the instructional 
innovation, if there is a change of instructor®? 

Answers: 

1. Nuciber or faculty members teAehing a given course and 
their- agreement on changes, * 

2. Degree or -amount of positive reaction by students and 
an Increase in achievement by students. 
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3, Ability to secure and mainiiain needed facilities 

and/or equipment necessary to continue innovations. 

What evidence of attitudes toward the results of your work 

in instructional development have you observed on the part 

of students? 

Answers: 

1. Favorable comments on changes. 

2. Achievement evaluation was not done. 

3. Faculty member reports positive change in students. 

On the part of participating faculty? 

Answers: 

1. Majority favorable and appreciate the opportunity. 

2. One member of faculty used experience to establish a 
major revision of oourse involved. 

3. Promotion received by one faculty member due in part to 
ISD involvement. 

4. Participating faculty devoted outside time to project. 

5. Faculty member reported positive change in self. 

On the part of campus administration? 

Answers: 

1, Vice President has written of interest in evolving 
educational development program on basis of project 
accomplishment s . 

2. Meetings are scheduled to discuss permanent organization 
of Instructional development agency. 

On the part of members of the development teams? 

Answers: 

1. Three of four members are favorable and are interested 
in permanent establishment of the development process. 
The other member regards the process as too time 
consuming and strictly' promotion of media. 
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Instructional development work is regarded as an avenue 
for increased recogr'ltion. 

Acknowledgement that the process requires great sensi- 
tivity to and empathy with instructor's problems. 



CHAPTER V: 



CONCLUSIONS AND RECOMMENDATIONS 



A number of specific conclusions and recomoiendations 
pertaining to Instructional development have already been 
presented In the reports of activities at the four partici- 
pating universities. Accordingly the Investigators here 
restrict their comments to conclusions related to the stated 
objectives of the overall demonstration and evaluation 
project. In the order they appear in Chapter I, they are: 

1. To select and orientate Innovative Institutions 

to serve as working models of systematic Instructional 

planning which utilizes the entire range of never media . 

There is little question that a measure of the success 
enjoyed by the four Institutions In this project can be 
attributed to a store of experience accumulated from successful 
media Imp^imentatlon in the past. The selection of these 
particular Institutions was Influenced In part by the need to 
find universities regionally located for easy access to visitors, 
and in part by the ability ef :he Institutions to complete project 

activities within the deadline:; imposed by the relatively brief 
two-year term of this study. Even though the institutions in- 
volved in the project already possessed varying amounts of 
experience In analyzing Instructional problems and in^lementing 
the newer media in courses, the requirement that they use and 
refine a novel development model was sufficient to present a 
new challenge to their expertise. The project encouraged 
demonstration Institutions to (1) define their steps for instr- 
uctional development; (2) review, evaluate, and consolidate the 
media resources already available for Instructional programs: and 
(3) establish a coordinated specialist team for instructional 
development assistance to faculty. 

The Investigators can only conjecture as to the effect 
this project might have on institutions having a lesser history 
In media and Instructional development. 

It is probable that Institutions new to attempting 
specialist-assisted Instructional development may need to Invest 
more time In the introductory activities which precede the devel- 
opment procedures tested by this study. The decision process 
embodied by the development flowchart operates best when there 
exists a mutual trust In the abilities among faculty and special- 
ists. The project demonstration institutions were able to 
capitalize on confidence already established with faculty and 
move Into testing the development procedures almost Immediately. 
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2, To evaluate their demonstration method of disseminating 
information by gathering data relevant to the adaption 
of the developmental systems from visiting institutions . 

There is little reason to dispute the general value of 
demonstrations as a means for disseminating information. However, 
the investigators believe the experiences of this project under- 
score some limitations of the demonstration method which also 
is supported in vjritings by Brickell and others. That is, the 
demonstration method is of greatest usa in communicating 
Information about concrete materials and short-term processes. 

In the case of instructional development, a situation 
where deliberations are complex and often stretch over weeks 
or months, it is very difficult to convey itc many aspects to 
visitors. The dissemination activities of both the demonstrat- 
ion institutions and the coordination unit proved most effective 
when they were confined to discussing the issues surrounding 
the development activities and exhibiting materials resulting 
from the development process. Examples of information easily 
connnuaicated are printed and taped comments reflecting the 
attitudes and reaction of faculty members to the analysis of 
Instructional problems, the organization of the development 
team, graphic representation of the development steps, concrete 
examples of decision-making, instructional materials produced for use 
in courses, and budget information. Transcriptions of the 
actual faculty-specialist deliberative process did not get 
across as well. 

However, visitors who were experienced educators were 
able to infer a good deal from representative presentations and 
did not require literal reenactment of the development activities. 

3. To evaluate the (a) feasibility, and (b) reliability of 
this developmental system within institutions, first 
(hopefully) with a common course, and second, over other 
key courses . 

The measurement of feasibility and reliability in a 
strictly empirical sense was not seriously considered in 
designing the project. Investigators agreed early that the 
diversity of activity likely to result from the work at the 
demonstration institutions would make difficult the standard- 
ization of data for comparison. Instead, evaluation was keyed 
to achieving some consensus among the members of the Project 
Evaluacion Team who would have an opportunity to inspect both 
detailed and summarized data submitted by the demonstration 
institutions. Thus, their estimate of feasibility for the 
development system model was a judgement of general usefullness 
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for other Institutions, based on the results of trials at the 
four project institutions. Feasibility would have been best 
tested if two of the demonstration institutions differed widely 
in personnel and material resources, and yet succeeded equally 
in achieving substantial improvements in identical courses of 
study. 



In fact the four institutions did exhibit differences 
in organization and competencies available, but hardly enough 
to test the development procedures under extreme differences, 
e.g. an institution which had few media facilities or experience 
in implementing media in instruction. The variance in existing 
facilities and staff competencies represented by four project 
institutions resemble a large number of colleges and universities. 
Evaluators found little evidence to suggest that the develop- 
ment steps were not compatible with mrst settings in higher 
education. Indeed the changes made by the demonstration schools 
in the instructional development stepc were relatively minimal 
and did not reject the proposed sequence of the process. 

The reliability of the development process, i.e. its ability 
to deliver again and again in the same setting what it purports 
to do was a quality also difficult to measure. Only two courses 
were selected for treatment at each institution, and at the 
writing of this report work is incomplete in the second course 
at all four schools. The opportunity to measure reliability 
comprehensively suffered somewhat, when it was impossible to 
work in the same courses at all four project Institutions. 

The few common observations that may contribute to questions 
of relidblllty pertain to faculty attitudes and development team 
specialists progress in working with instructors. The fact that 
a common development model was proposed is generally credited by 
observers with facilitating development work. Under the cir- 
cumstances, the specialist knew better at what stage he was work- 
ing, what had been accomplished and what to prepare for next. 

The instructor too generally felt secure because of the identity 
the model gave their concerns, once only intiutively harbored. 

The model was described by one specialist as a "roadmap" for 
instructional development. 

4, To refine the system* s organization and operating 
procedures, both in light of above evaluation and 
the re-evaluation of a second trial . 

Refinement here refers to reordering development steps 
in a new sequence, increasing the specificity of development 
steps, altering step size and/or Introducing new ones. Some 
development specialists preferred to reduce the number of steps 
by consolidating some and deleting others. Only cne institution 
chose to alter the sequence of the steps. This was done simply 



by joining the initial and final steps of the flowchart to 
depict a circular or recycling process model. The other 
institutions found that introduction of previously omitted 
feedback loops in the flow of development activities accom- 
plished the same end as did the creation of a circular model. 

Despite the reported decision by development teams not 
to Introduce new steps to the flowchart, evaluators observed 
a number of short-cuts or enabling activities individual 
specialists used to progress in their work. The evaluators 
agreed that these ploys and strategies were tangential to the pro- 
cess and did not qualify as full-fledged development steps. 

They chose to call them the "heuristics" of instructional 
development. 

These procedures constitute a hidden agenda for the 
development team specialist and often spell the difference 
between success and failure in the operation of the flowchart 
development steps. 

Interestingly, some specialists found it practical not to 
reveal even the flowchart steps to the instructor. They tended 
to inhibit discussion of teaching problems by faculty members 
who felt most at ease talking in their own terms. 

5. To devise and build in a means for generating 

research hypothesis within the developmental system . 

and 

6. To prepare a prototype list of research hypotheses that 

have been generated by the working demonstration models . 

The success of efforts aimed at these two objectives fell 
below the level of the investigators expectations. They now 
believe that the successful pursuit of these two goals may 
require the investment of more time and resources than could 
be spared within the framework of this study. Actually, the 
development procedures generated a large number of questions, 
but only a few could be regarded as readily researchable 
hypotheses. In short, a good question does not qualify as a 
research problem, when it is unmanageable by known research 
technology. The Evaluation Team observed that few experimental 
techniques and designs now exist that are suitable for the 
problems generated by the development procedures. 

However, the effective question-producing operation implicit 
in the tested development steps could be considered as hypotheses- 
generating, if the Baycsian-type statistician were available to 
enter the scene. Unfortunately, few are available and none in 
this project. 
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7. 



To sponsor major symposia at DAVI and AASA conventions > 
the participants being members of the demonstration teams 
and project staff * 

In addition to dissemination efforts at the demonstration 
institutions, a number of presentations were made in general 
sessions of large professional meetings. These served different 
purposes and audiences than did the demonstration institution 
clinics. The programs gave substance to instructional devel- 
opment discussions which previously were largely hypothetical. 

In addition they spurred interest in practitioners to replicate 
and further test the development procedures involved in the 
project and visit demonstration clinics for more detailed in- 
formation. Finally, the evidence presented at these meetings 
sharpened the definition of terminology associated with media 
and instructional design efforts. 

8. To aid the participating demonstration institutions 
in preparing summary presentations which capitalize 
on the advantages of media application . 

All four demonstration institutions have produced film- 
recording and/or videotaping documentary narratives of their 
course development experiences. Generally, projects such as 
this one find the demand for information on findings lags the 
conclusion of the study by a period of six months to two years. 
In order to prepare for this eventuality and preserve data in 
their most concrete form, Syracuse University has undertaken to 
produce a brief filmed record of their work. The University of 
Colorado has already produced a videotape of development session 
and used it in their clinics. San Francisco State College like- 
wise is in the process of videotape production, while Michigan 
State University has elected to produce filmed excerpts for 
examples and documentation of instructional development. 
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^ CHAPTER VI: SUMMARY 



In the aftermath of unprecedented changes in physical size 
and enrollment of colleges and universities, some institutions have 

to assist academic departments and individual faculty members innovate 
teaching methodologies and materials. 

The United States Office of Education-sponsored study A 
Procedural and Cost Analysis Str . dy of Media in Instructional Systems 
Development (CS-3-> 16-030) conducted at Micf Agan State University from 
1963 to 1965 describes a number of cases where the inq>lenentation of 
newer media was pivotal in increasing the effectiveness of instruction 
in courses with large enrollments. In addition this study produced a 
hypothetical instructional development model to serve as a guide to 
faculty members in the analysis of problems of instruction and selec- 
tion of appropriate forms of the newer media of communications. The 
use of these development procedures is based on the employment of 
professional assistance provided by an instructional specialist, an 
evaluation specialist, and a media specialist. 

The faculty member working with these three specialists seeks 
to make an optimum choice in a series of decisions related to teaching 
strategy, evaluation methods, content selection, and newer media util- 
ization. The flowchart for these decisions appears in Figure 1, page 3. 
The limited opportunity this investigation provided to evaluate the 
worth of this hypothetical instructional development model was expanded 
through a new two-year US(^- supported study Instructional Systems 
Development: A Demonstration and Evaluation Project . This folJow-up 
Investigation was conducted during 1965-67 and involved four institut- 
ions, Michigan State University, the University of Colorado, Syracuse 
University, and San Francisco State College. The overall purpose of 
the new project was to refine by trial the development steps defined 
in the earlier study and in the process disseminate this expertise to 
neighboring colleges and universities. Specifically the project 
objectives were: 

1. To select and orient innovative institutions which are to serve 
as working models of systematic instructional planning which 
utilizes the entire range of newer media. 

2. To evaluate this demonstration method of disseminating infor- 
mation by gathering data relevant to the adoption of the devel- 
opmental system from visiting institutions. 

3. To evaluate the (a) feasibility, and (b) reliability of this 
developmental system within institutions, first (hopefully) 
with a common course, and second, over other 1cey courses. 
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4. To refine the system's organization and operating procedures, 
both in light of the above evaluation and the re-evaluation 
of a second trial. 

5. To devise and build in a means for generating research 
hypotheses within the developmental system. 

X\J p^CpdiLC d pX.QuOujfpd JLXOW aaj ^vrwaawox*^ waa%*w 

have been generated by the working demonstration models.. 

7. To sponsor major s 3 nnposia at DAVI and AASA conventions, 
the participants being members of the demonstration teams 
and project staff. 

8. To aid the participating demonstration institutions in pre- 
paring summary presentations which capitalize on the ad- 
vantages of media application. 

9. To produce a summary document which (a) describes the major 
steps taken by the demonstrating institutions in implement- 
ing a systematic approach to instructional planning; (b) 
contains the final form of the development system; (c) in- 
cludes diffusion data representing the number of visiting 
institutions and those which have shown evidence of system 
adoption; (d) offers a prescription for the curricula of 
substantive degree programs based upon role-task specifi- 
cation of system specialists; and (e) presents comparative 
cost data for instructional systems development at each of 
the demonstration institutions. 

Each of the four institutions organized instructional dev- 
elopment teams composed of appropriate specialists and selected 
courses of study for review and revision. The study required 
that the courses selected be in different ac.?demic departments 
and enroll at least 500 students per year. Short of these re- 
quirements, the institutions were free to select and deal with 
whichever faculty members appeared to have immediate need for 
assistance and were willing to experiment with the development 
steps. 



The other organization within the Project consisted ‘of a Co- 
ordinating Unit for overall administration and a Project Eval- 
uation Team. The Project Evaluation Team consisted of special- 
ists from three other institutions, who were given access to all 
data generated by the study and asked to judge the gereralizability 
and worth of the hypothetical model on the basis of experiences 
observed at the four institutions. Data on these development ex- 
periences was obtained by extensive tape-recording of faculty 
member- specialist deliberations, periodic written reports and pro- 
ject-wide review conferences. 
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The course of the investigation was further guided by an 
Advisory Conmittee composed of university administrators ex- 
perienced in institutional research, design of instruction and 
media services development. The results of the investigation 
are summarized as follows: 

1. Usefulness o f. the model instructional’ 
development flowchart 

All parties to the instructional development trials 
found the decision steps flowchart practical in 
analyzing and improving the teaching approach in 
the selected courses. It served as a good referent 
or “road map*'. In some cases specialists found it 
more effective to deal with the decisions required 
in the professor’s own terminology rather than the 
technical terms proposed in the model. 

2. Appropriateness of the sequence of instructional 
development steps in the model 

The Syracuse team preferred to view the flow of 
decision activities as a circular process, per- 
mitting Initiation of a "development cycle" from 
any step. The Colorado team foui.d it simpler to 
deal with fewer steps and proceeded to reduce the 
model to five major development phases. The se- 
quence remained essentially the same. Michigan 
State University and the San Francisco development 
teams were satisfied to employ the model as is. 

3. Dissemination of Instructional Innovation 
by the Demonstration Method 

All foul development teams regarded the demon- 
stration method, in this case on-campus clinics, 
as a relatively ineffective method for communicating 
the drawn-out process of instructional development. 
They felt the demonstration method served best in 
illustrating the "products" of the development pro- 
cess, i.e. instructional materials and physical 
facilities. The essential give and take of the 
instructor- specialist transfections could not be 
recreated for visitors j although some of this flavor 
was transmitted by permitting the instructor to have 
lengthy and candid discussions with visiting groups. 
Each demonstration institution is producing a film 
and/or videotape documentary of their work. 



Influence of Instruction al Development Project 
on Participating and Visiting Institutions 

It is too early yet to determine the effect the 
demonstration institution clinics had on visitors. 
Each development team has reported that definite 
steps have been taken on each campus to establish 
a permanent instructional development agency, similar 
in many respects to the project organization which 
conducted the trials of the hypothetical development 
procedures, Michigan State already provides support 
for such work through the Educational Development 
Progran and its operational facility the Instruct- 
ional Development Service comprised of the Learning 
Service, Instructional Media Center and Department 
of Evaluation Services. 
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